Computer architecture and process of patient generation, evolution, and simulation for computer based testing system

ABSTRACT

A computer implemented simulation and evaluation method simulates interventions to a patient by a user, and evaluates the interventions responsive to predetermined criteria and the interventions. The method includes defining a test area to evaluate the user to at least one of predetermined criteria and a user profile, selecting genetic information of the patient responsive to the test area, and generating a patient history responsive to the test area and the genetic information. The method also includes receiving at least one intervention input by the user, and evaluating the user responsive to the intervention and predetermined criteria.

RELATED APPLICATIONS

[0001] This application claims priority from U.S. provisional patentapplication entitled COMPUTER ARCHITECTURE AND PROCESS FOR PATIENTGENERATION, EVOLUTION, AND SIMULATION, Ser. No. 60/029,856, toRovinelli, et al., filed Oct. 30, 1996, the details of which are herebyincorporated by reference.

FIELD OF THE INVENTION

[0002] The present invention is generally related to a computerarchitecture and process for patient generation, evolution, andsimulation, and more particularly to a computer architecture and processfor patient generation, evolution, and simulation for a computer basedtesting system.

BACKGROUND OF THE RELATED ART

[0003] Medical certifying organizations have traditionally relied uponpaper and pencil cognitive examinations as a method for the assessmentof the candidate's medical knowledge. Traditional formats such asmultiple choice questions have well-defined operating characteristicsand reliability for examining cognitive knowledge capabilities. See, forexample, Stocking ML, An alternative method for scoring adaptive tests,Research Report RR-94-98, 1994, incorporated herein by reference.

[0004] However, these tools generally measure in only cognitiveknowledge. These methods provide only primitive ability to assess acandidate's problem-solving abilities. See, for example, Stillman P L,Swanson D B, Ensuring the clinical competence of medical schoolgraduates through standardized patients, Arch Int Med 1978, Vol. 147,pages 1049-52, incorporated herein by reference.

[0005] Several organizations have previously experimented withcomputer-delivery of clinical content and evaluation. In the late 1960sand 1970s, the Ohio State University developed a self-directedIndependent Study Program which utilized a “Tutorial Evaluation System,”for conveying curriculum content. See, for example, Weinberg A D, CAI atthe Ohio State University College of Medicine, Comput Biol Med 1973,Vol. 3, pages 299-305; Merola A J, Pengov R E, Stokes B T,Computer-supported independent study in the basic medical sciences in:DeLand E C (ed). Information Technology in Health Science Education,Plenum Press, New York, 1973, incorporated herein by reference.

[0006] Co-synchronously Dr. Octo Barnett's laboratory at theMassachusetts General hospital began development of clinicalsimulations. See, for example, Barnett G O, The use of a computer-basedsystem to teach clinical problem-solving, Computers in BiomedicalResearch, Academic Press, New York 1974;, Vol. 4, pages 301-19; BarnettG O, Hoffer E P, Famiglieti K T, Computers in medical education: presentand future, Proceedings of the Seventh Annual Symposium on ComputerApplications in Medical Care, IEEE Press, Washington, DC 1983, pages11-13, incorporated herein by reference. The clinical simulations usedthe MUMPS language.

[0007] At approximately the same time, investigators at the Universityof Illinois developed a simulation model known as (Computer-AssociatedSimulation of the Clinical Encounter, or “CASE”). See, for example,Harless W G, Farr N A, Zier M A, et al., MERIT—an application of CASE,Deland E C (ed), Information Technology in Health Science Education,Plenum Press, New York 1978, pages 565-69, incorporated herein byreference. This system was at one time considered by the American Boardof Internal Medicine (ABIM) as at least one component of arecertification process. Friedman R B, A computer program for simulatingthe patient-physician encounter, J Med Educ 1973, Vol. 48, pages 92-7,incorporated herein by reference. Research supported by the ABIMdemonstrated that a computerized examination system appeared feasible inprofessional evaluation/certification settings. Reshetar, R A, et al.,An Adaptive Testing Simulation for a Certifying Examination, presentedat the Annual Meeting of the American Educational Research Association,San Francisco, Calif., April, 1992, incorporated herein by reference.

[0008] Stevens and colleagues have also demonstrated the feasibility ofusing computer-based systems for testing problem-solving ability inundergraduate medical school curriculum applications. See, for example,Stevens R H, et al, Evaluating Preclinical Medical Students by UsingComputer-Based Problem-Solving Examinations, Academic Medicine 1989,Volume 64, pages 685-87, incorporated herein by reference. Sittig andcolleagues have also examined the utility of computer-based instructionin teaching naive users basic computer techniques such as “drag anddrop” and other computer operations. See, for example, Sittig D F, JiangZ, Manfre S, et al., Evaluating a computer-based experiential learningsimulation: a case study using criterion-referenced testing, ComputNurs; 1995, Vol. 13, pages 17-24, incorporated herein by reference.

[0009] We have determined that the above described medical assessmentprocesses suffer from two weaknesses: 1) test development requiresre-generation of an examination with new material on a recurring(usually annual) basis; 2) although multiple choice questionsdemonstrate reliable performance in measuring cognitive knowledge, theuse of this format for assessing clinical problem solving has not beensupported by the literature. Another system was developed at theUniversity of Wisconsin. This project served as the nidus for theComputer-Based Examination (CBX) developed by the National Board ofMedical Examiners (NBME). See, for example, Friedman R B, A computerprogram for simulating the patient-physician encounter, J Med Educ 1973,Vol. 48, pages 92-7; Clyman, Stephen G., Orr, Nancy A., Status Report onthe NBME's Computer-Based Testing, Academic Medicine 1990, Vol. 65,pages 235-41, incorporated herein by reference. NBME's CBX developmentproject has been in evolution for over a decade, and has demonstratedvalidity in examining professional degree candidates. See, for example,Solomon D J, Osuch J R, Anderson K, et al., A pilot study of therelationship between experts' ratings and scores generated by the NBME'scomputer-based examination system, Academic Medicine 1992, Vol. 67,pages 130-32, incorporated herein by reference.

[0010] However, we have determined that the CBX model suffers from theproblem that the clinical simulations are “hard-wired” in computersource code which must be re-coded for each new examination. Once thesimulation has been used widely, the examination contents are no longersecure, necessitating continuous cycles of new simulation development.

[0011] The expert system literature describes the evolution inevaluation and training systems. Early artificial intelligence/expertsystem work concentrated on “rules of thumb” or heuristics to representproblem-solving strategies identified by domain experts. See, forexample, David J M, Krivine J P, Simmons R., Second generation expertsystems: a step forward in knowledge engineering, in: David J M, KrivineJ P, Simmons R. Second Generation Expert Systems, Springer Verlag, NewYork, N.Y. 1993, pages 3-23, incorporated herein by reference. We havedetermined that these rule-based systems were necessarily constrained tonarrow domains, and that the knowledge contained in the rules wasdifficult to validate. Id.

[0012] In addition, early expert systems suffered from rapidly decliningperformance when exposed to circumstances outside narrowly defineddomains. See, for example, Davis R. Expert systems: where are we andwhere do we go from here, AI Magazine, 1983, Vol. 3, pages 3-22; SimmonsR. Generate, Test and Debug: A paradigm for combining associational andcausal reasoning, in: David M, Krivine J P, Simmons R., SecondGeneration Expert Systems, Springer Verlag, New York, N.Y. 1993, pages79-92, incorporated herein by reference. We have determined that thisphenomenon occurred at least in part due to interactions among the manyrules needed to define a domain. Recent work indicates that therobustness of such systems is enhanced by providing knowledge ofdifferent types. See, for example, Simmons R. Davis R., The roles ofknowledge and representation in problem solving, In: David M, Krivine JP, Simmons R., Second Generation Expert Systems, Springer Verlag, NewYork, N.Y. 1993, pages 27-45, incorporated herein by reference.

[0013] We have further determined that experts generally not only relateto one dimension of knowledge when defining a rule, but also rely uponexpansive knowledge of how systems work (i.e., physiology andpathophysiology in the medical domain) in performing real-worldproblem-solving. See, for example, Davis R., Expert systems: where arewe and where do we go from here, AI Magazine, 1983, Vol. 3, pages 3-22,incorporated herein by reference. This realization has led tore-thinking regarding structure of knowledge-based systems to reflectthe tasks such a system should accomplish, the methods the system shoulduse to accomplish the tasks, and the knowledge required to support thesemethods. See, for example, David J M, Krivine J P, Simmons R., Secondgeneration expert systems: a step forward in knowledge engineering, In:David J M, Krivine J P, Simmons R., Second Generation Expert Systems,Springer Verlag, New York, N.Y. 1993, pages 3-23, incorporated herein byreference.

[0014] We have also determined that knowledge-acquisition for suchsystems entails development of a model for the domain and instantiation(i.e., encode and enter needed information into the system's datastructure) of the model with information acquired from knowledge donors.See, for example, David M, Krivine J P, Simmons R., Second generationexpert systems: a step forward in knowledge engineering, In: David M,Krivine J P, Simmons R., Second Generation Expert Systems, SpringerVerlag, New York, N.Y. 1993, pages 3-23; Breuker J, Weilenga B., Modelsof expertise in knowledge acquisition, In: Gida and Tasso (eds), Topicsin Expert System Design: Methodologies and Tools, North HollandPublishing, 1989, incorporated herein by reference.

[0015] To obviate the above described weaknesses, we have determinedthat it is desirable to provide a computer-based testing project whichwill: 1) instantiate medical knowledge as object-oriented datastructures known as knowledge base of family medicine; 2) utilize themedical knowledge structures to create realistic clinical scenarios(simulated patients); and 3) assess the candidate's clinical problemsolving ability as the effective intervention in the clinical progressof the simulated patient through the selection of various actions madeavailable by the testing system.

SUMMARY OF THE INVENTION

[0016] The computer-based testing system described herein representsknowledge at multiple levels of complexity. For example, reactiveairways disease is represented as a series of health states: Normal(Non-reactive) Airways, Reactive Airways-Mild, ReactiveAirways-Moderate, and Reactive Airways-Severe. Each health statecontains identifiers which relate the particular health state toprecedents and antecedents (e.g., Normal Airways serves as the precursorhealth state for Mild Reactive airways disease, and Mild, Moderate andSevere Reactive Airways Disease represent target health states from theNormal circumstance.)

[0017] Each health state in turn has associated findings, and specificfindings. For example, the Normal Airways state, the Finding “Shortnessof Breath” is instantiated with the Specific Finding “No shortness ofbreath.” Similarly, other Findings such as Respiratory Function andSevere Asthma Attack Frequency are instantiated with correspondingnormal Specific Findings (Normal Respiratory Functions, and No SevereAttacks.) This representation transports to each new health state in amanner which we have determined to be analogous to diagnosis. See, forexample, Genesereth M., Diagnosis using hierarchical design models,Proc. National Conference on AI, 1982, incorporated herein by reference.

[0018] The computer-based testing system of the present inventionpartitions knowledge into fundamental types: Health States, Agents,Findings, Specific Findings and Patterns describe system behaviors andcharacteristics. Courses-of-Action describe human activities whichmodify and evaluate the health state information and characteristicsdescribed in the model. Subdivision of knowledge types in this mannerfacilitates the knowledge acquisition process. This subdivision alsopromotes multiple levels of knowledge abstraction, which enhances thesystem's ability to represent varying levels of complexity.

[0019] For example, in the Computer-Based Testing System, a pattern suchas incidence is further sub-divided into sub-patterns such as incidencein females versus males, and incidence in various racial/ethnic groups.

[0020] Multiple levels of abstraction and types of knowledge impose asubstantial knowledge acquisition challenge. Knowledge acquisitionincludes several possible methodologies, including direct questioning ofdomain experts/protocol analysis, see, for example, Ericsson KaA, SimonH A, Protocol Analysis: Verbal Reports as Data, MIT Press. Cambridge,Mass. 1984, incorporated herein by reference, psychometric methods, see,for example, Kelly G A, The Psychology of Personal Constructs, NortonPress, New York, N.Y. 1955, incorporated herein by reference, andethnographic methods, Suchman L A, Trigg R H, Understanding Practice:Video as a Medium for Reflection and Design, In: Greenbaum, J, Kyng M(eds)., Design at Work: Cooperative Design of Compute Systems, LawrenceEarlbaum Associates 1991, pages 65-89, incorporated herein by reference.

[0021] Advantageously, the Computer-Based Testing System of the presentinvention has included a blend of these approaches. Direct questioninghas been used in querying family practice physicians regarding theirknowledge of and approaches to specific knowledge domains (such asosteoarthritis). Additionally, knowledge acquisition has included accessto appropriate scientific literature, which functionally serves toprovide an ethnographic assay of actual practice. Knowledge acquisitionhas also entailed protocol analysis, both in terms of analyzing specificphysicians' problem solving methodologies and incorporating explicitclinical processes such as those presented in published clinicalguidelines (a specific example here is the otitis media with effusionguideline developed by the Agency for Health Care Policy and Research).

[0022] To facilitate development of such a system, the present inventionis divided into three components: the knowledge base, the patientsimulation generator, and the presentation system. The knowledge basehas been designed and represented as a series of entity-relationships.The model has several fundamental entities: Patient, Health States,Findings, Courses of Action, and Agents. These entities haverelationships of INTERACTS_WITH, CONTACTS, IS_RELATED, EXHIBITS, HAS,EXPOSED_TO, LEADS_TO, ASSOC_WITH, LINKS_TO, USES, IDENTIFY, MANAGE,ALTER, REVEAL, and EVALUATE.

[0023]FIG. 1 describes an overall or conceptual view of the entities andrelationships included in the model. Rectangles indicate entitiesbetween entities in the model. Hexagons indicate relationships. Solidlines indicate Medical Knowledge Relationships (e.g., a course of actionsuch as treatment with non-steroidal anti-inflammatory agents can modifyspecific findings such as pain in the patient with osteoarthritis.)Dotted lines indicate Simulation/Evolution relationships which definehow a particular domain simulation has proceeded.

[0024] The patient simulation generator of the present invention reliesupon a series of generation methods to instantiate patients forpresentation to the certification/recertification candidate. Theprocesses function to evolve the patient forward (to reflect progressionof the disease process and response to interventions) and backward intime (to create a past history for the patient.) To accomplish thesetasks, the system utilizes processes for:

[0025] 1. Content specification—these processes define the scope of thesimulation

[0026] 2. Patient generation:

[0027] Past History (“backward” generation)

[0028] Present and Future History (“forward” generation)

[0029] 3. Simulation processes (in addition to patient generation):

[0030] Interface processes (for presentation of the patient findingsdeveloped from generation processes.)

[0031] Book-keeping processes (for keeping track of candidates'responses and patient evolution)

[0032] The patient generation process proceeds on the basis of aspecific health state identifier (coded in the database as a name andSNOMED code) passed to the process at the start of the simulation. TheSNOMED International structured vocabulary is a versatile nomenclaturefor describing medical ideas. See, for example, Côté R A, Rothwell D J,Palotay J L, Beckett R S, Brochu L, editors, SNOMED International: Thesystematized nomenclature of human and veterinary medicine, 3rd ed.Northfield, Ill., College of American Pathologists, 1993, incorporatedherein by reference. This nomenclature allows one to make inferencesfrom the codes used to represent each idea. For instance, the codeF-37022 represents “retrosternal chest pain.” The first character, “F,”indicates that the code is from a broad class of ideas called functions.The next to digits, “37,” indicate that the code involves a refinementof the code F-37000, “chest pain, not otherwise specified.” Similarly,code F-37020 specifies “precordial chest pain.” The code F-37022 impliesthat retrosternal chest pain is a kind of precordial chest pain, whichis a kind of chest pain, which is a kind of function.

[0033] The generation process produces a complete patient descriptionwhich reflects the EXHIBITS, HAS, INTERACTS-WITH, EXPOSED-TO,IS-RELATED, and CONTACTS relationships described earlier. Thesegenerated entity relations are stored as a collection of recordsreferred to as the “White Board” data structures. The information inthese records serves as input to the patient evolution process, which inturn evolves the patient's health status and medical/personalcharacteristics as a function of the passage of time orphysician/examinee intervention.

[0034] The original patient generation process is generally called onceat the session's start; the system calls the evolution processesrepeatedly in response to time progression and physician action.

[0035] The first phase of patient generation entails development of thepatient's history outline. This outline describes the series of healthstates and risk factors the patient experienced to reach the currenthealth state, TS. To develop TS, the system first calls the procedureGenderRace, which establishes the patient's sex and racial/ethnicorigin. Next, the system establishes the patient's age and age at onsetthrough the OnsetAge procedure. The CreatePerson process then assignsthe patient a birth date and name.

[0036] Once the patient's age, sex, racial/ethnic origin, and age atonset of the condition have been established, the OutlineFirstStepprocedure defines the precursor states and risk factors which serve asthe substrata for evolving the patient to the current time and targethealth state. The OutlineGeneralStep procedure is then callediteratively until the patient has arrived at the current TS. Theseprocesses are described in greater detail below.

[0037] Logical and procedural knowledge in the database described as“reasoning elements” (RE) (for example, Bayesian network describing ageneration method, Bayesian network describing a treatment plan, and thelike), included in the generation methods described above, “shapeselectors” which describe distributions for the n patterns by whichhealth states evolve (patterns in turn are specified by findings andsubpatterns), and courses of action (COA) which represent possiblefurther diagnostic and management strategies which candidates mightselect.

[0038] The patterns and subpatterns are represented as probabilitydistributions (discrete and continuous as appropriate for particularfinding) specified through the knowledge acquisition process. At thebeginning of a simulation, random number generation is used to select a“master percentile” (MP) which then serves as the reference forselecting particular patterns, findings and subpatterns from theappropriate specified distributions. These selected patterns are queriedto provide description of specific findings such as hyperglycemia inresponse to physician/examinee requests for information which are in theform of “courses of action” for a particular health state (e.g.,hyperglycemia as a manifestation of diabetes.)

[0039] Once presented with the patient description (age, race, sex,clinical findings), the candidate then selects appropriate COA's forfurther evaluation and/or management of the patient's health state.Selection of an interventional COA invokes pattern modifiers whichevolve the patient's health state by implementing shape modifiers. Thesemodifiers act upon the initially selected health state patterns toredefine the patient's health state or findings (e.g., a COA of insulinadministration would alter the hyperglycemic finding specified in thehealth state descriptions for diabetes mellitus.)

[0040] As mentioned earlier, COA's also include options for furthertesting/diagnostic procedures. For example, the candidate might chooseto select a glycosylated hemoglobin evaluation; the COA process wouldaccess the pattern for glycosylated hemoglobin instantiated at thebeginning of the simulation but which might not be reported unlessspecifically asked for by the candidate.

[0041] A COA can modify the health state in which a patient exists atone point in time. When the candidate selects such a COA, the simulatedpatient evolves to a new health state patterns associated with the newhealth state in the knowledge base. In order to avoid “state explosion”,health states closely associated with each other are represented asparallel health states not as combined health state entities.

[0042] For example, the initially generated patient for a case ofosteoarthritis might demonstrate mild osteoarthritis. However, otherhealth states, such-as obesity, might influence the progress of thepatient's arthritis from mild to moderate or severe disease. To avoidcombinatoric health state explosion, we have implemented a concept ofparallel networks of health states. In this representation, anewly-generated patient will exhibit instantiated health state patternsfor the primary domain (in this case osteoarthritis) and for theparallel health states (obesity in this example) which influence theprimary health state's progress.

[0043] As shown in FIG. 2, osteoarthritis can progress over time fromthe normal state to mild, moderate or severe osteoarthritis. For thisparticular illness, progress occurs in one direction only;osteoarthritis does not regress once developed, but can stabilize at aparticular degree of severity. Obesity represents a parallel healthstate which can influence the progression of osteoarthritis. Mild,moderate, and severe obesity can influence this progress at differentrates: the model permits representation of greater impact for moresevere obesity states. Notice also that obesity can regress (e.g.,severe obesity can revert to moderate obesity, etc.).

[0044] Any one of a number of health states might exist which couldprogress independently of osteoarthritis. For example, the patient whohas osteoarthritis will frequently utilize non-steroidalanti-inflammatory drugs (NSAID's) for treatment. These agents canimprove the symptoms of osteoarthritis, but also impact on the parallelstate of peptic ulcer disease. Treatment with NSAID's can induce anulcer, which can then evolve either on the basis of physician/examineeintervention for it, and/or for the course and treatment for otherparallel health states, and time with the course and treatment ofosteoarthritis.

[0045] The computer based testing system's fidelity depends upon accessto a rich representation of health state-specific knowledge. Thisknowledge consists, as described above, in more detail below. Thetemplate includes a NAME for the health state and an associated SNOMEDcode. The template also includes specific descriptions of the FINDINGS,PATTERNS and SUBPATTERNS for these FINDINGS. The patterns andsubpatterns are stored as a series of time and value pairs. As anexample of such patterns, consider the example of Reactive AirwaysDisease (RAD). One finding of interest is the prevalence of RAD as afunction of age, sex, and race. The prevalence for this finding appearsin the knowledge base as collection of graphs illustrating thepopulation prevalence conditioned on age, sex and race. Likewise, datasuch as acute exacerbation rates are represented as event ratedistributions. The subpatterns also include information describing howvarious treatment modalities will modify the exacerbation rate and otherpertinent findings such as peak expiratory flow rates and symptoms suchas shortness of breath.

[0046] The present invention provides a prototypical process fordeveloping domain-specific knowledge. The template for each domainincludes, for example, the following hierarchy:

[0047] HEALTH STATE: {name assigned by the knowledge donor, e.g.,“Normal Airway Reactivity”}

[0048] SNOMED CODE: {appropriate SNOMED code}

[0049] PREVALENCE: {age-sex-race specific prevalence; represented aspattern}

[0050] INCIDENCE: {age-sex-race specific incidence; represented aspattern}

[0051] FINDING: {general name for set of findings, e.g., “Asthma AttackFrequency” in reactive airways disease}

[0052] Specific Finding: {description of specific instance of a FINDING;e.g., for the FINDING of asthma attack frequency, one specific findingis “No Attacks”, associated with “Normal Airway Reactivity”}

[0053] Each HEALTH STATE affects multiple FINDINGS, which in turn haveSpecific Findings appropriate for that FINDING in that HEALTH STATE.Data such as incidence, prevalence, and attack rates are represented asPATTERNS (graphical functions which support the patient generationsimulation processes). The information is collected in paper templateform, and then transferred into computer-readable format using, forexample, any standard Knowledge Acquisition (KA) tool to enter theinformation into an object-oriented database.

[0054] The KA “front end” may be developed, for example, in the VisualBasic® and Visual C++® programming environments. Courses-of-Action(COA), such as further evaluation and/or management strategies, areentered using a standard editor that creates text files describingappropriate evaluation/management steps to support the simulationprocesses. The COA editor may also be designed under the MicrosoftVisual environments mentioned earlier.

[0055] The knowledge acquisition step includes the followingsubcomponents:

[0056] A. Health state specification

[0057] B. Enumeration of FINDINGs for the health state, and agreementamong the development team members

[0058] C. Population of templates with knowledge

[0059] D. Entry of health state knowledge into knowledge base using KAtool and/or direct high level pseudo-coding

[0060] E. Debugging, including generating multiple simulations, to testsystem stability/credibility

[0061] F. Validation including review of generated cases byrepresentative groups of family physicians

[0062] It is a feature and advantage of the present invention is to: (1)allow testing at remote sites and convenient times; (2) uniformly testan expanded range of important family practice activities, with fewerquestions on exotic problems; (3) adapt tests to examinees' responses orneeds; and (4) create reasonable questions at test sites to simplifyadministrative, economic, and especially security issues.

[0063] It is another feature and advantage of the present invention toprovide an approach that does not incur high maintenance costs andproduces efficient and affordable scenarios for a computer-based testingsystem.

[0064] It is another feature and advantage of the present invention toprovide a formal model of family medicine to achieve a relevant andrealistic implementation of a computer based examination.

[0065] It is another feature and advantage of the present invention toprovide an examination that does not require replacement with newquestions in order to preserve security of the certification process.

[0066] It is another feature and advantage of the present invention toprovide a computer based testing system that may measure problem-solvingcapabilities.

[0067] It is another feature and advantage of the present invention toprovide a computer based testing system that relies upon a knowledgebase of family practice which contains “patterns” and “subpatterns”which depict in probabilistic terms disease/condition incidence,prevalence, evolution over time, and response to interventions.

[0068] The present invention is based, in part, on our discovery thatprior computer based testing systems suffer from various problems,including the problem that the clinical simulations are “hard-wired” incomputer source code or static data structures which must be re-coded orreinstantiated for each new examination. Accordingly, in prior artcomputer based testing systems, once the simulation has been usedwidely, the examination contents are no longer secure, necessitatingcontinuous cycles of new simulation development.

[0069] The present invention is also based, in part, on our realizationthat the computer based testing system needs to be capable ofefficiently generating new patient cases for each candidate examined,and capable of effectively testing a candidate's problem-solvingability. We have discovered that the above may be accomplished using aknowledge base of family practice which contains “patterns” and“subpatterns” which depict in probabilistic terms disease/conditionincidence, prevalence, evolution over time, and response tointerventions.

[0070] To achieve the above features and advantages, as well as otherfeatures and advantages that will be apparent from the detaileddescription provided below, a computer implemented simulation andevaluation method simulates interventions to a patient by a user, andevaluates the interventions responsive to predetermined criteria and theinterventions. The method includes defining a test area to evaluate theuser on at least one predetermined criterion, selecting geneticinformation of the patient responsive to the test area, and generating apatient history responsive to the test area and the genetic information.The method also includes receiving at least one intervention input bythe user, and evaluating the user responsive to the intervention andpredetermined criteria.

[0071] In accordance with another embodiment of the invention, acomputer system and computer readable tangible medium is provided thatstores the process thereon, for execution by the computer.

[0072] In accordance with another embodiment of the invention, acomputer readable tangible medium is provided that stores an objectincluding the entity relationship model thereon, for execution by thecomputer.

[0073] There has thus been outlined, rather broadly, the more importantfeatures of the invention in order that the detailed description thereofthat follows may be better understood, and in order that the presentcontribution to the art may be better appreciated. There are, of course,additional features of the invention that will be described hereinafterand which will form the subject matter of the claims appended hereto.

[0074] In this respect, before explaining at least one embodiment of theinvention in detail, it is to be understood that the invention is notlimited in its application to the details of construction and to thearrangements of the components set forth in the following description orillustrated in the drawings. The invention is capable of otherembodiments and of being practiced and carried out in various ways.Also, it is to be understood that the phraseology and terminologyemployed herein are for the purpose of description and should not beregarded as limiting.

[0075] As such, those skilled in the art will appreciate that theconception, upon which this disclosure is based, may readily be utilizedas a basis for the designing of other structures, methods and systemsfor carrying out the several purposes of the present invention. It isimportant, therefore, that the claims be regarded as including suchequivalent constructions insofar as they do not depart from the spiritand scope of the present invention.

[0076] Further, the purpose of the foregoing abstract is to enable theU.S. Patent and Trademark Office and the public generally, andespecially the scientists, engineers and practitioners in the art whoare not familiar with patent or legal terms or phraseology, to determinequickly from a cursory inspection the nature and essence of thetechnical disclosure of the application. The abstract is neitherintended to define the invention of the application, which is measuredby the claims, nor is it intended to be limiting as to the scope of theinvention in any way.

[0077] The above objects of the invention, together with other apparentobjects of the invention, along with the various features of noveltywhich characterize the invention, are pointed out with particularity inthe claims annexed to and forming a part of this disclosure. For abetter understanding of the invention, its operating advantages and thespecific objects attained by its uses, reference should be had to theaccompanying drawings and descriptive matter forming a part hereof,wherein like numerals refer to like elements throughout, and in whichthere is illustrated preferred embodiments of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

[0078]FIG. 1 is a diagram describing an overall or conceptual view ofthe entities and relationships in the model used in the computer basedexamination system of the present invention;

[0079]FIG. 2 is a diagram describing the progression of osteoarthritisover time from the normal state to mild, moderate or severe states ofosteoarthritis;

[0080]FIG. 3 is a detailed diagram of the family medicine model,including the major entities, relations and modifying relations;

[0081]FIG. 4 is a flowchart of the overall process for the computerbased examination system of the present invention;

[0082]FIG. 5 is a flowchart of the history outline process whichgenerates the patient history in the computer based examination systemof the present invention;

[0083]FIG. 6 is a flowchart of the history generation process whichfinds values for the patient history in the computer based examinationsystem of the present invention;

[0084]FIG. 7 is a flowchart providing an overview of the stochasticprocess in accordance with another embodiment of the computer basedexamination system of the present invention;

[0085]FIG. 8 is a flowchart illustrating a first step in tracingprevious health conditions to generate past medical history of thepatient for the stochastic process of the computer based examinationsystem of the present invention;

[0086]FIG. 9 is a flowchart illustrating a second step in tracingprevious health conditions to generate past medical history of thepatient for the stochastic process of the computer based examinationsystem of the present invention;

[0087]FIG. 10 is an illustration of the entity-relationship model datastructure stored in the white board database when patients are notpre-generated;

[0088]FIG. 11 is an illustration of a modified entity-relationship modeldata structure stored in the white board database when patients are notpre-generated;

[0089]FIG. 12 is an illustration of parallel network structures for thecomputer based examination system of the present invention;

[0090] FIGS. 13-14 are detailed flowcharts of the process of thecomputer based examination or assessment system of the presentinvention;

[0091]FIG. 15 is an illustration of a main central processing unit forimplementing the computer processing in accordance with a computerimplemented embodiment of the present invention;

[0092]FIG. 16 illustrates a block diagram of the internal hardware ofthe computer of FIG. 15;

[0093]FIG. 17 is a block diagram of the internal hardware of thecomputer of FIG. 16 in accordance with a second embodiment; and

[0094]FIG. 18 is an illustration of an exemplary memory medium which canbe used with disk drives illustrated in FIGS. 15-17.

NOTATIONS AND NOMENCLATURE

[0095] The detailed descriptions which follow may be presented in termsof program procedures executed on a computer or network of computers.These procedural descriptions and representations are the means used bythose skilled in the art to most effectively convey the substance oftheir work to others skilled in the art.

[0096] A procedure is here, and generally, conceived to be aself-consistent sequence of steps leading to a desired result. Thesesteps are those requiring physical manipulations of physical quantities.Usually, though not necessarily, these quantities take the form ofelectrical or magnetic signals capable of being stored, transferred,combined, compared and otherwise manipulated. It proves convenient attimes, principally for reasons of common usage, to refer to thesesignals as bits, values, elements, symbols, characters, terms, numbers,or the like. It should be noted, however, that all of-these and similarterms are to be associated with the appropriate physical quantities andare merely convenient labels applied to these quantities.

[0097] Further, the manipulations performed are often referred to interms, such as adding or comparing, which are commonly associated withmental operations performed by a human operator. No such capability of ahuman operator is necessary, or desirable in most cases, in any of theoperations described herein which form part of the present invention;the operations are machine operations. Useful machines for performingthe operation of the present invention include general purpose digitalcomputers or similar devices.

[0098] The present invention also relates to apparatus for performingthese operations. This apparatus may be specially constructed for therequired purpose or it may comprise a general purpose computer asselectively activated or reconfigured by a computer program stored inthe computer. The procedures presented herein are not inherently relatedto a particular computer or other apparatus. Various general purposemachines may be used with programs written in accordance with theteachings herein, or it may prove more convenient to construct morespecialized apparatus to perform the required method steps. The requiredstructure for a variety of these machines will appear from thedescription given.

Best Mode for Carrying Out the Invention

[0099] The computer-based testing system described herein representsknowledge at multiple levels of complexity. The computer-based testingsystem of the present invention partitions knowledge into fundamentaltypes: Health States, Agents, Findings, Specific Findings, Patterns andSub-patterns describe system behaviors and characteristics.Courses-of-Action describe tasks and methods used to apply, modify, andevaluate the health state information and characteristics described inthe model. Subdivision of knowledge types in this manner facilitates theknowledge acquisition process. This subdivision also promotes multiplelevels of knowledge abstraction, which enhances the system's ability torepresent varying levels of complexity.

[0100] For example, reactive airways disease is represented as a seriesof health states: Normal (Non-reactive) Airways, Reactive Airways-Mild,Reactive Airways-Moderate, and Reactive Airways-Severe. Each healthstate contains identifiers which relate the particular health state toprecedents and antecedents (e.g., Normal Airways serves as the precursorhealth state for Mild Reactive airways disease, and Mild, Moderate andSevere Reactive Airways Disease represent target or sequential successorhealth states from the Normal circumstance.)

[0101] Each health state in turn has associated findings, and specificfindings. For example, in the Normal Airways state, “Asthma AttackFrequency” appears as a Finding which is instantiated with the SpecificFinding “No attacks.” Similarly, other Findings such as RespiratoryFunction and Severe Asthma Attach Frequency are instantiated withcorresponding normal Specific Findings, (Normal Respiratory Functions,and No Severe Attacks.) This representation transports to each newhealth state in, what we have determined to be somewhat analogous todiagnosis.

[0102] Advantageously, the Computer-Based Testing System of the presentinvention in the knowledge acquisition process uses direct questioningin querying family practice physicians regarding their knowledge of andapproaches to specific knowledge domains (such as osteoarthritis).Additionally, knowledge acquisition has included access to appropriatescientific literature, which functionally serves to provide anethnographic assay of actual practice.

[0103] Overview of Testing/Recertification Process

[0104] The testing and/or recertification process, for example, unfoldsas follows. After initial certification, examinees initiaterecertification software on workstations on computer systems. Theexaminee begins recertifying at any convenient time and could suspendthe examination at the conclusion of any simulated patient encounter.The software of the present invention presents a patient by using text,illustrations, still pictures, and video. The examinee questions andexamines the simulated patient, reaches conclusions about the situation,and suggests treatment options. The simulated patient may expresspreferences about these options.

[0105] After receiving a treatment plan, the patient leaves, maybefollows the plan, and perhaps later returns for follow-up. In themeantime, the examinee sees other simulated patients. To discouragecheating, the software offers so many cases that a diplomate observinganother examinee recertify gains little advantage with regard to testcontent.

[0106] The present invention maintains records of the informationgathered, the hypotheses entertained, and the recommendations made foreach patient. After monitoring performance on several similar cases (forinstance, cases involving diagnosis and management of adult-onsetdiabetes mellitus), the program draws conclusions about the physician'sability to handle this class of problems. If competence has beendemonstrated, the class of problems may be removed from furtherconsideration for several years. Until competence has been demonstrated,the physician receives feed-back on specific areas for improvement andcontinues to see cases from this class of problems.

[0107] The testing and/or recertification process could eventuallybecome a continuous learning experience at the office or home. Somerecertification activities might qualify as continuing medicaleducation, partially offsetting the time needed to recertify. Examineescould anticipate failure to recertify and take corrective measures yearsbefore actually failing.

[0108] The present invention provides an approach that does not incurhigh maintenance costs to maintain efficient and affordableexaminations. The present invention also provides a formal model offamily medicine to achieve a relevant and realistic implementation ofthis kind of computer-based examination.

[0109] In general, a model describes the kinds of information that couldbe collected regarding a topic. For instance, a model of a mailingaddress should include at least a name, street address, apartmentnumber, city, state, and ZIP code. A database built upon this modelcould list these items for each entry. Not every item in the modelshould be described for every entry in the database; many addresses haveno apartment number. Incomplete database entries still provide usefulinformation; even if a street address is missing, the city to search canbe found.

[0110] Finally, the model limits what the database could do; it couldnot easily list first names. A model of diagnostic medicine of thepresent invention includes diseases, historical and examination data,and links between diseases and data. These models represent knowledgethat physicians apply to uncertain or imprecise cases. The addressexample suggests a list of simple observations, called a database. Adiagnostic program uses a collection of more abstract information, suchas a statistical summary of a database, to draw inferences about asingle case. The program and its information are often called aknowledge base.

[0111] We have determined that a well-designed formal model supportsautomatically created case simulations, reducing the long-term cost ofwriting cases by hand and improving security. The formal model of thepresent invention considers that medicine is full of diagnosticcomplexities including disease interaction. Thus, diabetes could changethe severity of pain experienced during an acute myocardial infarction.With this information, the knowledge base of the present invention isable to support a realistic simulation process—a simulated diabetichaving an acute myocardial infarction will experience a specificdiscomfort. The present invention attempts to carefully defineinteractions for a number of health states that constitute the bulk offamily medicine.

[0112] We have further determined that diagnosis and patient managementare inextricably linked to time. Time receives relatively littleattention in many knowledge bases and is often summarized verysuccinctly. For instance, a knowledge base might describe “chest painlasting more than 30 minutes” as a symptom of acute myocardialinfarction. This knowledge base could misinterpret 29 minutes of chestpain as evidence against acute myocardial infarction, and 2 years ofchest pain as an indicator of acute myocardial infarction. The presentinvention also supports the related concepts of continuity of care andobservation.

[0113] In addition to these problems, family physicians deal with a hostof issues that we have determined are not routinely modeled indiagnostic software. Most of these issues reflect the overwhelmingimportance of patient management in family medicine.

[0114] First, family medicine occurs in a social context that is oftenignored in computer-generated simulations. Knowledge bases do not modelsocial interactions or family structure.

[0115] Second, family practice patients arrive with attitudes shaped byexperience, and physicians must adjust their strategies to cope withthose attitudes. Adjustments range from changing interview style toaltering treatments. Variability in patient attitudes limits thelikelihood that there exists one best answer for groups of patients withsimilar medical conditions.

[0116] Third, family physicians are not so much engaged in diagnosis asin helping patients improve the length and quality of their lines.Family physicians spend considerable time reassuring worried patients,alleviating symptoms, and preventing the onset or progression ofdisease.

[0117] We have determined that the final testing and/or recertificationproblem, evaluating the responses of diplomates, also requires a modelof what family physicians do. All dichotomous evaluations, especiallypass-fail tests, use arbitrary standards. The challenge is to setstandards using generally agreeable and meaningful criteria. The presentinvention provides the flexibility to determine to whom the criteriashould be agreeable—certainly to diplomates, but perhaps also topatients, insurers, or other customers. Specifying these customers willhelp establish meaningful criteria for certification decisions.

[0118] For instance, diplomates have an interest in maintainingrespected credentials, patients want effective care, insurers desire lowcosts, and public health advocates have an interest in clinicalguidelines. It is not at all clear how to respond to these diverseinterests. The present invention delivers flexible models to describethe consequences of family practice activities, as seen by variousparties, so that board certification remains a pertinent processregardless of changes in the health care system.

[0119] We have determined that a model is needed to describe the scopeof family medicine in epidemiologic terms, while including theinformation about individual variation that differentiatesindividualized patient care from public health. The model will be thefoundation of a family practice knowledge base storing data about familymedicine. The model also supports other applications of benefit tofamily physicians. Specific software applications might involve medicalrecords, structured vocabularies, medical reference tools, decisionsupport systems, and continuing education programs.

[0120] Data structures to describe the activities of family physiciansinclude a series of entity-relation diagrams. In an entity-relationdiagram, entities usually represent things (nouns). The relations(verbs) illustrate how the entities interact. For instance, anentity-relation diagram of an address list might have an entity called“person,” and an entity called “place,” connected by a relation called“is at.” One could read this diagram, “person is at place.” The personentity would store people's names, the place entity would storeaddresses, and the “is at” relation would describe when and why thisperson is at that place. Thus, a person could now live at one place,previously live at another place, and continuously work at the firstplace. One person, two places, and three “is at” relations describe thisaddress history. This address model is flexible and realistic.

[0121] We have determined that an important class of events exist in themodel of family medicine, which we call “modifying relations,” ormodifiers. In database terms, modifiers are relations betweentraditional relations. Modifiers extend the conventional entity relationdiagram and provide a means of managing statistically dependent events.

[0122] Model Structure

[0123] The family medicine model includes the major entities, relationsand modifying relations shown in detail FIG. 3. Formal concepts in themodel are capitalized throughout the text. The model emphasizesdiagnostic and management issues, variability in populations, and time.It describes consequences of anatomic and physiologic processes, butlargely omits anatomic and physiologic reasoning as such. It is alsocapable of describing interpersonal relationships and is expendable toinclude an explicit representation of families or communities.

[0124] Modifiers (for example, Bayesian network from a Lead to relation,Bayesian network describing risk factors for progression, and the like)are relations that might change values in other relations. Dynamicentities and relations contain information relevant to patientsimulations. Dynamic information for an individual patient is derivedfrom data in other dynamic and static entities and relations. Thedynamic Record entity has relations mirroring the Population'srelations. Static entities and relations contain the best availablemedical knowledge, similar to data in medical literature.

[0125] The following major entities appear in the design: Populations,Records, Health States, Findings, Courses of Action, and Agents ofChange.

[0126] Populations represent real humans; their relations shouldprecisely describe all data that physicians consider. Populations can belarge groups with a shared characteristic, such as white males orsingle-parent families. An individual patient is a Population of 1; apregnant woman is a Population of 2; a nuclear family with 2 childrenand 2 parents is a Population of 4.

[0127] Records model beliefs about people; a Record's relationssummarize inferences about a Population. If a parent brings an infant tothe office, this design represents the infant as a Population, theparent as another Population, and the parent's description of the infantas a Record. The physician can obtain historical information about theinfant from two sources: the physician's medical Record of the infant,and the parent's Record of the infant. The physician can obtain currentobjective information by examining the infant as a Population. The datalinked to Populations are absolutely precise, but can be observed, if atall, only during medical encounters. Records summarize the history ofthose real data imprecisely and potentially inaccurately.

[0128] Populations have Records of themselves, modeling a patient'sself-image and memories. As with other Records, a patient's self-Recordsummarizes historical information with variable accuracy and might bethe physician's only source of some historical information.

[0129] A Population is primarily a list of relations with otherentities. A Record not only lists relations with other entities, butalso defines encounters during which these relations were discovered. ARecord can contain conflicting data acquired at different encounters.

[0130] Health States include all normal health states; classic diseasepresentations; early, subtle, or late disease presentations; and somedisease combinations. Health States also include groups of Health Stateswith shared characteristics, such as cardiovascular diseases anddiseases of glucose intolerance. The SysteMetrics Corporation publishesDisease Staging Clinical Criteria, which define numerous stages in thedevelopment of diseases. See, for example, Gonella J S, Louis D Z, GozumM E, editors, Disease staging clinical criteria, 4th ed. Ann Arbor,Mich: MEDSTAT Systems, 1994, incorporated herein by reference.

[0131] Each of these stages represents a distinct Health State entity inthis design. The SysteMetrics staging of diabetes mellitus defines stage1.1 as asymptomatic diabetes, stage 1.2 as symptomatic diabetes, stage1.3 as type I diabetes mellitus, and stage 2.1 as diabetes withend-organ damage. Each of these stages defines at least one Health Stateby the presence of specific objective criteria.

[0132] Stage 2.1 might be divided into a group of Health Statesrepresenting each damaged end organ. To represent multiple end organdamage, one might simply superimpose these states.

[0133] Findings include genetic, physiologic, symptomatic, physical, andtest-generated data, and clusters of such data. For instance,musculoskeletal chest pain could be a Finding. This Finding could be anexample of a Finding called chest pains, which would represent all kindsof chest pains. Chest pains could be an example of a still largerFinding called symptoms. Findings are defined by a collection of one ormore Features, whose current value can be described by a number on ascale. One Feature pertinent to pain is severity, which might bedescribed on a 10-point scale.

[0134] Structures called Patterns describe the possible values of eachFeature over time. A Pattern typically lists a series of values andcorresponding percentiles at several points in time. Pediatric growthcharts are the most widely used real example of Patterns. A blank growthchart illustrates at least the following observations: (1) Normal birthweights vary within a narrow range. (2) Weight increases relativelyrapidly in the first few months and years. (3) The absolute variation inweight (e.g., the difference between 90th and 10th percentile weights)increases after birth. (4) Most people reach a fairly constant weight byearly adulthood. A pattern listing 10th and 90th percentile weights forpeople at age 0, 1 year, 2 years, and so on, illustrates the sameconcepts.

[0135] Growth charts also predict future values from past information. Achild at the 50th percentile for weight now is expected to stay near the50th percentile. If this child later reaches the 5th percentile ofweight, the expected pattern is absent. The ensuing diagnosticevaluation is an effort to account for the deviation by finding a weightPattern that explains all observations. These concepts extend easily tomany other values, such as temperature. People have an averagetemperature of about 37° C., but some are a little cooler and some alittle warmer. Normal temperature fluctuates within a narrow rangeduring a lifetime, and most deviations from that range are consideredabnormal.

[0136] Another example would be ST segments on a electrocardiogram.Following an acute myocardial infarction, ST segments usually rise byvarying amounts, fall, and return to normal. The ST segment deviationfrom base line varies with time and can be described by a Pattern,similar to the variation in weights of growing children.

[0137] Many values change in predictable ways. Patterns might havecycles, sub-Patterns, and sub-sub-Patterns to describe these changes.The average value of a variable often changes during a lifetime, whilethe instantaneous value depends on a combination of annual, lunar, andcircadian cycles. For instance, a nonpregnant 20-year-old woman shouldexperience predictable lunar and circadian temperature fluctuations.

[0138] Sub-Patterns also describe consequences of other events, such astaking a drug. For instance, a dose of acetaminophen might lower a feverfor 4 hours. A fever responsive to acetaminophen could be modeled by ahigh-temperature Pattern with a sub-Pattern indicating 4 hours of normaltemperatures following acetaminophen doses. A person experiencing thisfever and taking acetaminophen every 4 hours maintains a normaltemperature. A physician observing this temperature Pattern would needto halt the acetaminophen to distinguish between a normal temperatureand fever responsive to acetaminophen.

[0139] Sub-Patterns characterize Features and therefore Findings. Forinstance, one of the chest pain Findings might be “crushing substernalchest pain relieved by rest or nitroglycerin and exacerbated byexertion.” This description implies a Finding with a designatedlocation, a “crushing” Feature with some pattern, and 3 sub-Patternsdescribing the effect of rest, nitroglycerin, and exercise. The clinicalappearance of simulated patients with this Finding might still vary,depending on the allowed variation in sub-patterns. For instance, painmight be more quickly relieved by nitroglycerin than rest or vice versa.

[0140] Finally, Patterns include Shape Selectors that help maintainconsistency between variables. Shape Selectors are an example ofReasoning Elements, for example, small programs loosely based on thestructure of Arden syntax medical logical modules. See, for example,Johansson B G. Wigertz O B, An Object oriented approach to interpretmedical knowledge based on the Arden syntax, Proc Annu Symp Comput ApplMed Car, 1992, pages 52-56, incorporated herein by reference.

[0141] Reasoning Elements define variables; assign their values fromdata about the simulation; use loops, “if . . . then” statements,equations, and random numbers to reach conclusions; and finally producesome output. In Findings, the Shape Selector produces one percentilecurve to represent the values of a Feature in an individual patient. Forinstance, although pediatric growth charts allow considerable variationin normal height and weight, one child will exhibit a precise series ofvalues for both height and weight. Height will closely track onepercentile curve, as will weight. The percentile of the height curveoften limits the possible percentiles of the weight curve: healthychildren at 95th percentile height rarely exhibit 5th percentile weight.Most children follow a weight percentile equal to the height percentile±20. The weight Shape Selector can use this equation to restate thefamiliar height-weight growth chart.

[0142] Patterns model time and are one approach to interrelated medicalobservations. Time affects most numeric values in the model.Consequently, Patterns appear in nearly every entity and relation.Patterns describe the incidence of diseases at different ages, thelikelihood of diseases progressing with time, and concentrations ofdrugs. Courses of Action (COA) represent people's activities. Not onlycan these activities be medical, such as taking a blood pressure orperforming a coronary artery bypass graft, but they can also includeattending school, working, asking and answering questions, and followingadvice.

[0143] Populations invoke Courses of Action to decide when to visit aphysician, how to answer questions, and whether to follow advice.Therefore, Courses of Action may advantageously be written to includemissed appointments, lying to physicians, and ignoring physician advice.These actions could even depend on aspects of the physician's conduct,such as how the physician chooses to obtain information.

[0144] Courses of Action have complex internal structures. A Course ofAction organizes Step, which gather, process, and modify informationabout Populations or Records. For example, a Step might be to obtain ablood pressure from a person. Each Step uses a Reasoning Element toaccomplish its tasks. In the case of obtaining a blood pressure, theReasoning Element would determine and report the simulated patient'ssystolic and diastolic blood pressure.

[0145] A group of Steps that can occur in any sequence is called aBatch. For example, when checking both right and left arm bloodpressures, the order in which the arms are checked is probablyunimportant, so these can be distinct Steps within a Batch. The Courseof Action lists a series of Batches that must be executed in sequence,and describes any mandatory delays between Batches.

[0146] For example, to check orthostatic blood pressures, recumbentpressures would be obtained in one Batch. The patient would sit or standin a second Batch. After a short mandatory delay, sitting or standingpressures would be obtained in a third Batch. Courses of Action alsodescribe possible earnings, costs, pleasure, and discomfort thatmotivate people to seek or avoid activities.

[0147] Agents include physical, chemical, biological, behavioral, andsocial events capable of influencing health States or Findings. TheseAgents can be therapeutic, injurious, or both. Agent descriptionsinclude data about intake, metabolism, and excretion, as applicable. Forinstance, a long-acting steroid is a chemical agent. Followingintramuscular injection, the steroid will have predictable local andsystemic concentration Patterns as the chemical dissipates from theinjection site. The steroid might be metabolized to other compounds andexcreted. Exposure to Agents normally occurs during a Course of Action,as this example illustrates.

[0148] The model of Agents describes their recognition, their presence,and the presence of metabolites or byproducts. Other parts of the model,such as the sub-Patterns of Findings, describe the effects of Agents.

[0149] Table 1 lists relations shown in FIG. 3. The Health States Leadto Health States relation describes how diseases evolve, and istherefore, critical for simulations. Preventive medicine scenarios mightuse this relation to generate patients who would benefit from screening.Case management problems can use this relation to model both the pastand evolving history of a patient.

[0150] Table 1. Relations Between Entities

[0151] Population Contacts Population

[0152] Population Related to Population

[0153] Population Interacts with Courses of Action

[0154] Population Exposed to Agents of Change

[0155] Population Has Health States

[0156] Population Exhibits Findings

[0157] Agents of Change Cause Health States

[0158] Health States Lead to Health States

[0159] Findings Associated with Health States

[0160] Findings Link to Findings

[0161] Course of Action use Agents of Change

[0162] Courses of Action Identify Agents of Change

[0163] Courses of Action Treat Health States

[0164] Courses of Action Alter Findings

[0165] Courses of Action Reveal Findings

[0166] Courses of Action Evaluate Findings

[0167] Note: These relations link entities in the model together.

[0168] Unlike traditional knowledge bases, this relation links Findings(with their Patterns) to a Health State, rather than linking a range ofFinding values to a Health State. Sensitivity and specificity arerepresented as age dependent Patterns, rather than constants. Thesensitivity of a Finding will be lower and the specificity higher inthis model than in traditional knowledge bases.

[0169] The Findings Link to Findings relation describes causalassociations between Finding Patterns, such as “severe cough causesabdominal muscle pain.” This relation contains data about causality,mechanisms, and temporal constraints. This relation facilitatesreasoning about Findings.

[0170] The Courses of Action Treat Health States relation illustratesmeans of curing Health States or preventing their progression.Treatments therefore modify probabilities in a lead to relation.

[0171] Courses of Action have three relations with Findings. The first,Alter, implies changing a Feature Pattern by invoking a sub-Pattern. Forexample, giving acetaminophen could alter a fever. The second relation,Reveal, links examining Courses of Action to the Findings they produce.For instance, a procedure called “taking a blood pressure” revealssystolic blood pressure. The third relation, Evaluate, links a Findingto a Course of Action that might be used to investigate its cause. Thisrelation would link a Finding of systolic hypertension to a Course ofAction describing its work-up.

[0172] The Population Contacts Population relation traces transmissionof communicable Agents and potentially beliefs. Population Is Related toPopulation describes biological and social relations and the history ofthose relations, and traces transmission of genetic Agents. These tworelations allow descriptions of arbitrarily defined families, witharbitrarily harmonious interactions.

[0173] The Population Interacts with Courses of Action relationdescribes why the Population began the Course of Action, what theCourses of Action cost interested parties, and how comfortable thePopulation was during the Courses of Action. This model allows a patientto remember an unpleasant experience and resist having it repeated.Because Courses of Action can include negative (buying a therapy) orpositive (receiving a paycheck) change in wealth, this relation is alsocapable of being used to model patients' economic inability to followmedical advice.

[0174] The Population Exposed to Agents of Change relation describesperceptions about the exposure, knowledge of exposure, and the course ofAction responsible for the exposure. This relation can describe exactlyhow an Agent was distributed in, metabolized by, and excreted from thisPopulation.

[0175] The Population Has Health States relation includes the precedingHealth State, a list of Findings attributable to the Health State, andthe age at onset, diagnosis, and evolution of the Health State. HealthStates affect different individuals in different ways, and treatmentoften depends on the patient's impairments and perceptions.Consequently, a patient's beliefs about disease progression andperceptions of a Health State belong in the Has relation.

[0176] The Population Exhibits Findings relation has similar perceptionattributes. Perceptions can be divided into Dysutility and concern.Dysutility indicates a trade-off a patient would accept to return tonormal. Concern indicates a trade-off a patient would accept for fullreassurance that a Finding or Health State does not portend futureDysutility. For instance, a patient with a minor left-sided chest painmight rate its current Dysutility as $5 (“I would spend $5 to relievethis pain for today.”), and the concern as $100 (“I would spend $100 forassurance that nothing serious caused this pain.”). If the pain persistsunchanged, both of these values might decline as the patient learns tocope with the discomfort and becomes confident that the symptom has noprognostic importance. Thus, patients can have changing attitudes aboutstable conditions. Patients would typically seek medical care whenprovoked to so by a Dysutility or concern.

[0177] Records have the same relations as Populations, except that thedetails are always more ambiguous, inaccurate, or both. For instance, apatient might have influenza starting December 15, while his Record ofhimself indicates that he developed influenza between December 10 andDecember 13. The patient's Record of himself is both ambiguous (thereare 4 possible days of onset) and incorrect (none of the days isDecember 15).

[0178] We have further determined that the data described in the Leadto, Associated with, and Link to, relations often change with medicalinterventions or other events. Modifiers describe events that cause apermanent variation in the expected history of these relations. Forinstance, an event might make evolution to another Health State more orless likely (regular low-dose aspirin reduces the risk of acutemyocardial infarction), or could permanently alter the likelihood ofexhibiting a finding (cardiac transplant prohibits myocardial ischemicpain). The dashed lines in FIG. 3 show Modifiers. The following examplesillustrate some modifiers (for example, Bayesian network from a Lead torelation, Bayesian network describing risk factors for progression, andthe like).

[0179] Population Interacts with Courses of Action modifies HealthStates Lead to Health States. An appendectomy alters the progression ofacute appendicitis to appendiceal rupture. For example,life-span-altering interventions always modify a Lead to relation.

[0180] Population Exhibits Findings can modify Health States Lead toHealth States. For example, being overweight increases chances ofdeveloping a deep vein thrombosis or pulmonary embolism.

[0181] Population Has Health States can modify Health States Lead toHealth States. Diabetes accelerates the onset of cardiovascular disease.

[0182] Population Has Health States can modify Findings Associated withHealth States. Diabetic neuropathies diminish pain associated withmyocardial infarction or extremity injuries.

[0183] Modifications of these relations account for many benefitsascribed to receiving medical care. Other benefits can occur whenmedical interventions temporarily decrease the severity o Findings.

[0184] The model described herein is intended to be a highly structuredand realistic representation of family medicine that will guide thedesign of the family practice knowledge base and support the generationand evaluation of recertification examinations. In this model, thefollowing are strong assumptions: (1) Health States are discrete anddistinguishable on the basis of associated Findings, which are alsodiscrete and distinguishable on the basis of the Patterns of theirFeatures. (2) After choosing a percentile curve in a Pattern torepresent some value, the percentile does not change substantially. (3)Changes in Patterns (e.g., the probability of one Health State evolvingto another) can be described for important combinations of risk factors,interventions, and time of occurrence. (4) Transitions from one Patternto another can be estimated by simple means. (5) Modifying relations donot have important interactions with one another. (6) Highly developedanatomic and physiologic models are not necessary, because associationsbetween Findings provide the same information.

[0185] Although the model should have clear places to store nearly allinteresting facts about family practice, test generation does notrequire a comprehensive description of all facts used in familypractice. The proposed test generates plausible problems from a set ofdata intentionally skewed to generate interesting (i.e., discriminating)cases. The present invention provides the flexibility to avoidcontroversial questions by controlling skewed data. For instance, if themanagement of borderline diabetes is controversial, the presentinvention allows editing of the family practice knowledge base so thatdiabetics' fasting blood glucose levels are always markedly elevated.The family practice knowledge base would then be incapable of creating aborderline diabetic.

[0186] The diagram of the model illustrated in FIG. 3 reflects manyfamily medicine concepts, and therefore, helps students, physicians andothers understand the process at work in family medicine. For instance,the diagram illustrates that Populations have biological and socialrelations. Populations exist in Health States, which evolve into new,sometimes undesired Health States.

[0187] A major goal of family medicine is to retard or stop undesirableevolutions and promote desirable evolutions. Stopping one undesirableevolution could, however, result in a different undesirable evolution.In addition, physicians who treat symptoms will Alter Findings, but donot necessarily Treat Health States. Altering Findings usually changescurrent quality of life, whereas treating Health States usually changesfuture quality and quantity of life. Because Findings occur in thecontext of Health States, we have determined that physicians contemplatewhat Health States might be responsible for Findings, rather than Alterthe Finding without considering future quality of life. The only toolsavailable for these causes are Courses of Action. Physicians prescribeCourses of Action, but only patients Interact with Courses of Action.For example, the prescription does not guarantee that the patientfollows the correct Course of Action. Agents (e.g., drugs) make adifference only when used in the context of a Course of Action.

[0188] The model's details provide further insights for students. First,time is an extremely important element of primary care. Patterns becomemore distinctive as time passes, simplifying diagnosis. The total riskof going from one Health State to another increases with time,increasing the value of early interventions. Second, patients havevariable and evolving attitudes about Health States, Findings, andCourses of Action. The goal of medicine might not be to adhere to anendorsed Course of Action, but to optimize each patient's perception ofhis or her quality of life. To reach this goal, physicians adjustCourses of Action to accommodate individuals' attitudes. Third, theimportance of time and attitude in optimizing the quality of a patient'slifetime suggests that continuity of care might help some patients.

[0189] The scope of family practice and the importance of protocols,time, individual variations and attitudes, and rationales distinguishesthe content of the family practice knowledge base. That is,advantageously, some differential diagnosis of internally generatedcases is possible using the model.

[0190] In this model, differential diagnosis largely depends onestablishing the presence of Findings, which in turn depends onestablishing the presence of Patterns and sub-Patterns of Features.Except in rare cases of pathognomonic values, confidence in the presenceof a Pattern will increase with the passage of time.

[0191] We have also determined that the structure of an interface tomedical reference systems might be enhanced using the model. Currentreference systems use the structure of medical publications and lists ofabstracted subject headings to facilitate searches through very largedatabases. These searches can yield large numbers of extraneouscitations, especially for novice users.

[0192] The model suggests an alternative indexing strategy, as well as agraphical search interface. For instance, one could view a queryinterface similar to FIG. 3. To request a query about the effect ofinsulin treatment on the development of retinopathy in diabeticpatients, one selects diabetes from an unrestricted list of HealthStates. The Lead to allows the user to select diabetic retinopathy froma list of diseases restricted to diabetic sequelae. The Modifierspecifies which Course of Action or Agent of Change to consider. Thecomputer delivers a list of references mentioning insulin in a diabetesLeads to diabetic retinopathy relation. Searching for a particularrelation between two entities improves the efficiency of searchesusually performed by naming the entities.

[0193] Overview of Patient Generation/Evolution Processes

[0194] We describe here an overview of processes used in thecertification/recertification system. The processes are divided intofour main groups:

[0195] 1. Patient generation processes:

[0196] history outline processes

[0197] history generation processes

[0198] 2. Simulation processes

[0199] Presentation interface processes

[0200] Patient evolution processes

[0201] Patient generation processes are called once and produce thesubject for the examination session. Simulation processes may be calledrepeatedly several times. The patient generation process presents thepatient to the examinee, collect the examinee's responses and queries,and evolve the patient. See FIG. 4 for a pictorial overview of thesystem.

[0202] For the patient generation process, we assume that the area forthe simulation—a specific object, say A, from the class AREA—and ahealth state, say H, from the primary network of the area A are given.For example, A may be the area of the adult onset diabetes and H may bethe health state of symptomatic diabetes.

[0203] The patient generation process consists of two phases:

[0204] 1. history outline, and

[0205] 2. history generation.

[0206] The goal of the history outline phase is to generate aprogression of health states and risk factors traversed by the patienton the way from the normal condition to the specified health state H. Itstarts with a call to the procedure that establishes sex and race of thepatient being generated (referred to as procedure GenderRace). The nextstep establishes the age of onset of H (call to procedure OnsetAge).

[0207] The goal of the next step is to select the precursor state forthe target state in the simulation as well as risk factors(circumstances) that will affect the patient under construction. Thiswill be accomplished by a call to the procedure OutlineFirstStep.

[0208] The next procedure, OutlineGeneralStep, is called iterativelyuntil the normal health state is reached. In each iteration, it findsthe precursor health state as well as its onset time. When the normalhealth state is reached, the history outline phase is complete. See FIG.5 for a flowchart of this process.

[0209] The GenderRace procedure generates sex and race of the patientunder construction.

[0210] CreatePerson creates a basic description of the person. We selectlast, first and middle names, and age of the person, as well as twobasic demographic findings: sex and race. These last data are stored asEXHIBITS tuples (since demographic findings are treated as findings).

[0211] The OutlineFirstStep procedure generates the precursor state forthe target health state for the simulation, and its onset age. Inaddition, it selects circumstances to which the simulated patient hasbeen subject. This procedure also creates an object HS_path, stored onthe white board and containing the sequence of HAS instances for theprecursors of TS, starting with the normal health state and ending withTS. This sequence will be used later in the history generation phase.

[0212] The Generating history outline, and more specifically, theOutlineGeneralStep procedure, generates the complete path of precursorsof the target health state. It starts in the normal health state andterminates in the target health state TS (of course, the last but onestate on the path has already been generated by OutlineFirstStepprocedure).

[0213] History Generation

[0214] The history generation phase finds values that are established ineach case when they differ from normal (normal values are derived fromthe defaults maintained in the knowledge base). The general outline ofthis phase is given in FIG. 6.

[0215] The reasoning element, called generation method, describing how agiven health state or a risk factor determines a finding, plays animportant role in this phase. The generation method either provides adescription of all relevant basic features at all relevant sites (fornormal states), or determines which basic features at what sites need tobe adjusted and by what specific findings. The main input for this phaseis the list of associated objects attached to the object P of typePERSON (the object of the simulation).

[0216] The history generation process looks at all associated objectsand modifies values of patterns describing relevant basic features sothat the detailed description of the patient is consistent with thehealth state history as created in the earlier phase. Therefore, in thisphase we focus on describing findings and their basic features. To thisend, we look at all health states represented by HAS instances. We sortthem according to their onset times. This results in a list in which allstates normal in their areas precede all the abnormal states. The reasonfor this is that all normal states start at time 0. For each of thesenormal states we will run its generation method. This creates a list offinding names and site names to which the findings pertain, and definesthe domain of all findings for which specific descriptions are created.

[0217] Next, for every finding, the patterns of its basic features areinstantiated. We obtain these patterns from “normal” specific findingbelonging to the finding in question. To select specific curves, we usea percentile value. This value will generally be selected from, forexample, the range [0.15, 0.85] uniformly at random. Each time we needto use this value to select a specific pattern, we modify it, forexample, by a randomly selected number from the range [−0.05, 0.05]. Inthis fashion the modified value is, for example, in the range [0.10,0.90].

[0218] After all normal states are processed, patterns of all basicfeatures of all relevant findings are instantiated for life. From nowon, when processing other health states these patterns are modified. Theidea is to run the generation method for a health state. As a result weget a list of sites and basic features which must be modified as well asspecific findings where the new patterns can be found. If only somesites for the finding are generated, only those sites need to bemodified. To modify the patterns, we use patterns captured by theappropriate specific finding. Again the basic percentile is varied andused in the selection. The selected pattern is then superimposed on theexisting pattern (its values replace the old values starting with theonset age for the health state).

[0219] The generation method associated with the health state H,generates the list of relevant findings with additional information onsites and specific findings. That is, for each finding we maintain thelist of sites and with each of those we associate the list of all basicfeatures (names) corresponding to the finding. Finally, these basicfeatures are described by their patterns.

[0220] The PatientDescription procedure selects HAS instances. It thenarranges them according to onset times, generally earliest first. Inthis process, the procedure invokes the generation method procedures foreach health state, thus creating EXHIBITS tuples describing findingsassociated with health states.

[0221] The InitPt Description (Initialize Patient Description) procedureinitializes the list PATIENT FINDINGS, which contains all findingsrelevant to the primary health state as well as all secondary(modifying) health states. It creates all corresponding EXHIBITSinstances and attaches them to the list associated_objects. All thesefindings are initialized to their normal values.

[0222] After the call to InitPtDescription, the domain of findings,sites and basic features, which subsequently will be modified, isdefined. CreatePtDescription scans the list of HAS instances and adjustsfindings so that the resulting patterns are consistent with the historyof health states.

[0223] Patient Evolution

[0224] As explained earlier, we assume that data required by theprocesses is stored in the entity relationship model, white board (WB)and in the area of memory local to patient generation and evolutionprocesses. This local memory will be denoted as LM. We start theevolution phase with the patient fully described and stored in the WB.An equivalent description exists in LM. Several HAS instances describecontinuing health states (one of them—primary). After the assessmentphase (requiring physical examination and history taking) the examineeproposes treatment consisting of one or more courses of action. Thesecourses of action may alter some of the health states the patient iscurrently in. All selections made by the examinee are gathered in atable coa_list.

[0225] LEAD_TO data describes probabilistic information on progress fromone health state to another. This data depends on modifiers. At present,we use a small generic set of modifiers: “fast progress”, “moderateprogress” and “slow progress”. For each of these modifiers, and for anedge in the health state network between a precursor health state PS,and the target health state TS, the entity relationship model containsan estimate of the flow along that edge.

[0226] Courses of action are represented in WB by a table whichdescribes their structure in terms of elementary courses of action. Wewill describe this structure below. In addition, each course of actioncontains a reasoning element. This reasoning element, given an edge (apair (PS,TS)) and a set of other current health states (as modifyingevents), computes one of these three modifiers. Flows on the edgesstarting in the current health state are used in the selection process.Once the selection is made, duration risk stored in the appropriateLEAD_TO tuple is used to determine the onset time for the selectedhealth state.

[0227] The following structure is used to represent a course of actionCOA in WB. The data is stored in a table with, for example, four columns(additional columns may be necessary later for evaluation purposes). Thefirst column is labeled ECOA (elementary course of action). It lists allconcrete elementary courses of action that might be used in aconstruction of COA. The second column describes the type of thecorresponding elementary course of action. ECOAs of the same type areidentified by the same integer in the second column. The third columncontains one of five boolean operators: none (NOR), single (XOR), atleast one (OR), some but not all (NAND), all (AND). All members of atype are assigned the same operator in column 3. The fourth columncontains weights which are used in the matching process.

[0228] One of the courses of action listed with every health state iscalled TIME. It describes the effects of no specific action by theexaminee and serves as a default course of action.

[0229] The evolution phase is accomplished by the procedure calledEvolve. Evolve has three input parameters: patient P, patient's age Tand the list coa_list of COAs selected by the examinee. Evolve starts bycreating the list of patient P continuing health states. This isaccomplished by the procedure called SelectPresentHas. SelectPresentHasselects from the list of P associated objects those HAS instances thatrepresent continuing health states. It arranges selected HAS instancesin a list.

[0230] For each health state PS described by the list of selected HASinstances, we then identify in all the courses of action that arerelevant to PS. It gathers all those courses of action that are inrelation MANAGE with the health state PS, in the list called, forexample, coas.

[0231] At this time, the closest COA, among those found relevant to PS,to the examinee selection (described, recall, by the list coa_list) ischosen. For the course of action, say COA, target states are created forPS, corresponding modifiers and flows. This data is used for evolution.

[0232] These steps are repeated for each health state PS. When theprocess is completed, all successor health states are represented bymeans of the corresponding HAS instances. The evolution step iscompleted with a call to CreateDescription procedure. It generatesdescriptions of specific findings corresponding to the health states.

[0233] Stochastic Process For Patient History Generation

[0234] The present invention provides a method to automated authoring ofmajor events in simulated medical histories. We have designed aknowledge base with temporal descriptions of the incidence andprevalence of health conditions and plausible intervals between healthconditions. Each health condition is part of a small sequence of relatedand mutually exclusive health conditions. Many of these small networksexist in parallel.

[0235] We have determined that a patient's overall health can bedescribed by a vector indicating the patient's current health conditionin each network. A patient's location in one network often affectstiming of transitions in other networks. The knowledge baseadvantageously uses modifiers (for example, Bayesian network from a Leadto relation, Bayesian network describing risk factors for progression,and the like) to describe the influence of these and other risk factors,as well as interventions, on incidence and transition times. Astochastic history outlining algorithm uses these data to construct alifetime and recent medical history whereby a patient might develop aspecified vector of health conditions.

[0236] The present invention generates a large number of plausiblehistory outlines. The present invention automates the authoring of majorevents in the lives of simulated patients. The present invention appliesa Monte Carlo process to multiple stochastic trees, to generate largenumbers of plausible case outlines. Further automated embellishment ofthese outlines yields complete, usable simulated case histories.

[0237] Previous efforts to simulate patients from data have usedsensitivity information stored in a diagnostic database, or QuickMedical Reference, to stochastically create a description of findings ina patient with a disease. See, for example, Bergeron B. Iliad: ADiagnostic Consultant and Patient Simulator, MD Computing 1991, Vol. 8,pages 46-53; Miller R A, Masarie F E, Myers J D, “Quick MedicalReference(QMR)” for diagnostic assurance, MD Computing 1986, Vol. 5,pages 34-49, incorporated herein by reference. However, we havedetermined that these simulations lack rich historical details and maygenerate implausible combinations of events. See, for example, SumnerW., A review of Iliad and QMR for primary care providers, Archives ofFamily Medicine 1993, Vol. 2, pages 87-95, incorporated herein byreference.

[0238] Some simulations generate patient details from a complete andprecise mathematical model of pathophysiology. See, for example,Valdivia T D, Hotchkiss J, Crooke P, Marini J., Simulating the clinicalcare of patients: A comprehensive mathematical model of humanpathophysiology, Proc 19th Annu Symp Comput Appl Med Care. 1995, page1015, incorporated herein by reference. This elegant approach isfeasible in intensive medical care and some restricted organ systems,but primary care problems are not so well understood at present, andtherefore require empirical description.

[0239] Accordingly, we have also developed a process for generatingdetailed patient histories culminating in a specified set of simulatedhealth problems. The first segment of the algorithm creates an outlineof the medically important events in a patient's life, including thepatient's age at the onset and termination of different healthconditions or exposures to biologically active agents. The secondsegment of the algorithm yields a detailed description of continuouslydefined facts about the patient, such as physical and chemicalcharacteristics, morphology, function, and sensations throughout life.

[0240] The history outlining algorithm essentially creates paths throughtemporally reversed Monte-Carlo processes, casting major events in apatient's history while guaranteeing that the history ends withspecified medical conditions. See generally, Rubinstein R Y, Simulationand the Monte Carlo Method, New York, N.Y., John Wiley and Sons Inc.;1981, incorporated herein by reference. This process is applied to a setof stochastic disease history models, each describing the evolution ofone health problem.

[0241] A knowledge base stores-these models, along with standardmodifiers that calculate temporal constraints on disease progression,conditioned on comorbidities and treatments. This algorithm is capableof generating many plausible cases in a short period of time precedingan examination.

[0242] The “Health condition Leads To Health condition” cycle is thecentral component in the generation of a patient history. A healthcondition is a named collection of facts, which usually have prognosticimplications. Typically, the facts that connote a health condition havea specified degree of variation from normal ranges, and are thought toarise from a common underlying cause. A health condition can usually beconsidered to be located at one or more body structures where thatunderlying cause is present.

[0243] Health conditions uses patterns and subpatterns to predict theirprevalence and incidence, conditioned on factors such as sex and race.Prevalence and incidence are provided in a widely used structure calledshape, which plots a value over time. In this situation, time indicatesthe simulated patient's age.

[0244] A health condition uses a generation method reasoning element toestablish the facts pertaining to its instantiation. These facts mayinclude events like drinking alcohol or driving cars, but most facts arespecific instantiations of more generic medical concepts, such assymptoms or laboratory values, in specified body parts. For instance,the generic concept of “synovial fluid glucose level” might beinstantiated as “normal” in “both knees.” Shapes describe exactly how avalue in this instantiation may reasonably evolve or fluctuate overtime.

[0245] Two special classes of health conditions exist. First, normalhealth conditions are incident only at birth (or conception, dependingon testing goals). Second, “Alive” is a health condition whoseprevalence shows the proportion of a cohort that survives to any age.The age specific prevalence and incidence of all other health conditionsare defined as the percentage of living individuals at that age whoexperience or acquire the condition, respectively.

[0246] The leads to relation connects one health condition (theprecursor) to another (the target), and describes possible timeintervals required for evolution from the precursor to the target. APattern describes a probability density function (pdf) of these timeintervals, conditioned on comorbidities, treatments, and other riskfactors. This duration pdf provides a time constraint mechanism. Forinstance, a duration pdf for the progression of mild to moderate kneeosteoarthritis, given obesity, might indicate a probability density ofzero in the first five years following the onset of mild osteoarthritis,a uniform probability density from year five to year twenty, and then aprobability of zero. This implies that all simulated obese patientsdevelop moderate osteoarthritis between five and twenty years after theonset of mild osteoarthritis, and forbids simulated onsets at othertimes.

[0247] The modifiers of a Lead to relation also provide time constraintsfor risk factors. This allows the model to represent the concept thatobesity must exist for a period of at least 10 and up to 40 years forthis duration pdf to apply.

[0248] Finally, the Lead to relation provides information about howquickly and completely to convert from the findings typical of theprecursor to findings typical of the target. For instance, if each kneeosteoarthritis stage is a health condition, and each stage has a typicaldegree of joint space narrowing, then the transition from one stage toanother should be accompanied by more narrowing of the joint space. TheLead to relation can indicate that this narrowing occurs over years, andthat the narrowing is nearly complete when the simulation asserts thatthe latter osteoarthritis stage is present.

[0249] A series of Lead to relations connect health conditions intosmall networks illustrating evolutionary sequences of events. Thesenetworks often suggest a disease staging scheme, such as (Stage O) NoKnee Osteoarthritis, (Stage 1) Mild Knee Osteoarthritis, (Stage 2)Moderate Knee Osteoarthritis, and (Stage 3) Severe Knee Osteoarthritis.

[0250] We call this sequence a parallel health condition network. It is“parallel” to many other networks of health conditions that existsimultaneously in a person. In general, a parallel health conditionnetwork lists transitions that occur among an exhaustive set of mutuallyexclusive health conditions occurring in one body part. For instance,the left knee of a patient exists in one of the health conditions in theosteoarthritis network. The right knee also exists in one of theseconditions, but not necessarily the same condition found in the leftknee. The patient simultaneously exists with one condition in a gastriculcer network, a weight network, and numerous other networks.

[0251] A simulated patient's overall medical condition is therefore avector, V, listing the current health condition from each parallelnetwork at each involved site. A case specifies vector V₀, indicatingthe health conditions instantiated at the initial presentation of asimulated patient, and sufficient information to create a history ofvectors culminating in V₀.

[0252] Most of the parallel networks in any given case are inactive.These define an initial, usually normal, (stage 0) condition of theparallel network. Most cases contain a few active parallel networks.Active networks presenting at stage 1 or higher represent active medicalproblems. Active networks presenting at stage 0 represent potentialproblems, such as complications resulting from an active problem or itstreatment. The examinee's task is generally to identify and respond toactive networks in advanced stages, while minimizing disease progressionin active networks at stage 0.

[0253] Active networks can be divided into two categories. A caseusually focuses on care for a primary network “P” (for instance,osteoarthritis of the knees). A comorbid network “C” usually includeshealth conditions that influence, or are influenced by, the stage ofevolution of a primary network. For instance, obesity is a risk factorfor osteoarthritis, and osteoarthritis may worsen obesity by limitingexercise. Comorbid networks that do not interact with the primarynetwork in any important manner may serve as distractors.

[0254] For instance, an episode of urethritis might be irrelevant toosteoarthritis, but suggests Reiter's syndrome as an alternativeexplanation for knee pain with an effusion. An active, stage 0 comorbidnetwork provides opportunities for complications. For instance, asimulated osteoarthritis patient presenting with a “No gastric ulcer”health condition could advance to “Gastric ulcer” after receivingsteroidal nonsteroidal anti-inflammatory drugs.

[0255] When an active parallel network describes a chronic condition,acute exacerbations may be expected with some of the health conditionsin the network. An exacerbation network “E” is a parallel networkdescribing acute flares of illness that occur during a more chronichealth condition. For instance, flares of knee pain with effusions mayoccur in patients with chronic osteoarthritis. In principle, healthconditions within an exacerbation network can have their ownexacerbations. The simulation process of the present invention allowsexacerbation networks to contain cycles, unlike primary and comorbidnetworks.

[0256] A simulated patient's medical history is the sum of the eventsculminating in the case defining vector, V₀. The case providessufficient information to create many plausible histories, but does notstore histories per se. Consider a case defined to culminate in severebilateral knee osteoarthritis and morbid obesity. The relative sequenceof events on the primary and comorbid networks are not necessarilyconstrained. Obesity might be required to occur before the onset of mildosteoarthritis. However, the onset of morbid obesity could occur beforeor after the onset of moderate osteoarthritis.

[0257] The Cartesian product of two active, linear parallel healthcondition networks, P and C, yields a two dimensional web of healthcondition combinations. This produce re-establishes the complexityavoided by the parallel network simplification, and calls attention tointeractions between P and C. A vertex in this web is composed of theith health condition in P and the jth health condition in C, and isrepresented by the vector V₀=(P_(l), C_(J)). Evolution can be assumed tooccur in only one dimension at a time. If evolution in both networks canoccur simultaneously in life, one can be assumed to occur first, and theother a moment later for purposes of the model. That is, the set ofvectors V⁻¹={(P_(i-1), C_(J))}; (P_(i), C_(J-1))} are immediateprecursors of vector V₀, but (P_(l-1), C_(j-1)) is not. Similarly, theset of vectors V⁻² includes (P_(l-2), C_(J)), (P_(l-1), C_(J-1)), and(P_(l), C_(J-2)).

[0258] Three kinds of interaction are possible in the web formed bynetworks P and C. First, the networks may be completely independent, sothat evolution along one dimension has no implications for evolution inthe other. Second, progression through one network may depend on theconcurrent condition of an independent network. For instance, theincidence of early osteoarthritis conditions is dependent on thepresence of obesity. Finally, mutually dependent networks create a webin which progression through each network depends on the concurrentcondition of the other network. For instance, a realistic simulation ofa severe osteoarthritis history might require modeling a “vicious cycle”where obesity accelerates osteoarthritis, which in turn acceleratesobesity.

[0259] The cartesian product of N parallel health condition networkssimilarly yields an n-dimensional web of health condition combinations,with potentially complex interactions. Data acquisition for these websis a daunting task, but might be simplified by (1) limiting the numberof dimensions, (2) ignoring improbable health condition combinations,particularly when describing vicious cycles, and (3) assumingindependence for some kinds of test cases even when dependence exists inreality.

[0260] Stochastic Process History Outlining Process

[0261] The goal is to produce patient care scenarios for recertifyingdiplomates to manage. The data described above allow automaticgeneration of such cases, starting from a case specification. The caseis composed of primary network P, and comorbid health condition networkC. Network P is composed of health conditions P₀, . . . P_(n) and “leadto” relations PL_(0->1), . . . PL_(n-1→n). Network C is composed ofhealth conditions C₀, . . . C_(m) and “lead to” relations CL_(0→1), . .. CL_(m-1→m).

[0262] Chronic health condition P_(J) in network P has acute flaresdescribed by parallel network E. Network E is composed of conditions E₀,. . . E_(q) and “lead to” relations EL_(0->1), EL_(1->0), . . .EL_(q-1->q), EL_(q->q-1). The normal condition of network E is E₀, andthe network may cycle through E₀ up to X times.

[0263] The vector V₀=(P_(l), C_(J), E_(k)) summarizes the healthconditions required at the presentation of the case. Health conditionsP_(l) and E_(k) may be incident or prevalent at presentation. Incidenthealth conditions would typically require both diagnosis and management,while prevalent health conditions would often be known diagnoses, andrequire only management decision. Health condition C_(J) is usuallyprevalent.

[0264] The first step assigns the sex, race, and other geneticallydetermined facts to the prospective patient. If P_(i) is an incidenthealth condition in the simulation, the incidence pattern for healthcondition P_(l), is conditioned on sex and race. Sex and race areassigned by obtaining the area under the incidence curves for male andfemale patients of each race. The simulator makes a weighted randomselection of the patient's sex on the basis of the results.

[0265] In the weighted random selection process, a series of positivevalues is normalized to one by dividing each value in the series by thesum of the series. The resulting series defines a probabilitydistribution. To select an item according to this probabilitydistribution, the interval from zero to one is divided into consecutivesubintervals of lengths equal to the corresponding probability theseries. A random number from zero to one is generated from the uniformdistribution. The interval to which it belongs defines the selecteditem.

[0266] Because the incidence or prevalence of some illnesses, such asknee ostecarthritis, can increase dramatically with age, some correctionto approximate the absolute number of cases occurring at each age may beuseful, depending on the goals of the simulation. To obtain absolutenumbers of incident or prevalent cases at each age in a cohort, theincidence or prevalence at each age is multiplied by the fraction of thecohort in that age interval. Formula 1 illustrates this calculation, andthe general procedure for multiplying two shapes.

[0267] Formula 1. Absolute prevalence of health conditions as a functionof age:

[0268]  Absolute prevalence(P_(l), n)=prevalence(P_(i), n) *prevalence(Alive, n)

[0269]

[0270] Where prevalence (health condition, n)=the prevalence of healthcondition at age n years.

[0271] Similarly, the joint absolute prevalence of P_(i) and C_(j) canbe calculated by multiplying the absolute prevalence of P_(l) by theprevalence of C_(j) in each age interval. although the prevalence ofeither or both health conditions may be explicitly conditioned on thepresence to of the other, knowledge acquisition efforts are unlikely tocapture such dependencies. Calculating the joint prevalence reduces thechance of creating an unsolvable history, for instance by creating aprevalent case of P_(i) at an age where C_(j) does not exist, regardlessof the prevalence produced in knowledge acquisition. A weighted randomselection of an age of presentation can be made from the product of theage specific prevalence of all representing health conditions, and thespecial condition “Alive.”

[0272] Often, either P_(i) or E_(k) is an incident health condition, andthe age of onset of the presenting health condition vector, V₀=(P_(l),C_(j), E_(k)), is determined by the preceding step. In addition, theimmediately preceding health condition vector, V⁻¹, must be (P_(i-1),C_(J), E_(k)) if P_(l) is incident, because any other vector would makeP_(l) prevalent rather than incident at age N. More commonly, E_(k) isincident and vector V⁻¹ must be (P_(i), C_(j), E_(k-1)). Alternatively,if V₀ consist only of P_(i) prevalent health conditions, then the age ofonset of V₀ is unknown. In general, health condition vectors contain amixture of conditions with known ending times (e.g., precursors ofincident conditions in V₀) and unknown ending times (e.g., prevalentconditions in V₀).

[0273] Assume that P_(l) is an incident health condition at age N. Theinteresting vector is therefore V⁻¹=(P_(l-1), C_(j), E_(k)), becausehealth condition P_(l-1) evolved to P_(l) at age N. One possibleprecursor of vector V⁻¹ is P_(i-2), C_(J), E_(k)) which would evolve tovector V₋₁ at the age of onset of health condition P_(l-1).

[0274] The age of onset of P_(l-1) is constrained in part by the agespecific incidence of P_(l-1), and N. The incidence of health conditionP_(l-1), conditioned on race and sex yields the number of new cases peryear per number of persons at risk, in each year from birth to age N.Because the simulated patient must belong to a cohort of individuals wholived until age N, corrections to obtain an absolute incidence areusually not important.

[0275] The age of onset of health condition P_(i-l) is furtherconstrained by the plausible duration of P_(i-l). For instance, ifP_(i-l) always progresses to P_(i) within ten years, then a case ofP_(i-l) must have begun between ages (N-10) and N. The “lead to”relation PL_(i-1>i) provides a duration pdf, conditioned on pertinentfacts representing some known modifier. The duration pdf is aprobability distribution function defining probabilities of evolution toP_(l) time intervals subsequent to the development of P_(i-1). Theduration pdf is truncated at the time equivalent to the age ofpresentation, N (assuming that P_(l) could not have begun before birth),and reversed in time. The reversed duration pdf indicates at age 0 theprobability that a transition from P_(i-l) to P_(l) would take N years,the simulated patient's entire life. In the year before presentation, atage N-1, the reversed duration pdf shows the probability that thetransition would occur after exactly one year.

[0276] For each year from birth to the age of onset of P_(i), theincidence of health condition P_(l-1) and the reversed duration pdf aremultiplied to obtain a weighting factor for the onset of P_(i-1) in thatyear. These weights are used to make a random weighted selection of oneyear to propose as the age of onset for the health condition P_(i-1).This age represents one proposal for the age of onset of V⁻¹=(P_(i),C_(j), E_(k)).

[0277] Formula 2. Weight (W_(n)) for establishing the onset of healthcondition P_(l-1) at age n:

W_(n)=Incidence(P_(l-1), n) * DurationPDF (P_(i-1), N-n)

[0278] Where:

[0279] N=age of onset of health condition P_(i) DurationPDF (healthcondition, x)=probability that health condition evolves to its successorduring the time interval x-1 to x years after its onset.

[0280] In general, this procedure is repeated for each health conditionwith an onset time after birth (or conception) in the currentlyinteresting vector, V⁻¹. The result is a proposed list of ages of onsetfor a subset of vectors in the set V⁻². The next step proposes ages ofonset for the remaining vector in V⁻².

[0281] Assume that health condition C_(J) is a prevalent condition in asimulated patient presenting at age N. Assume that the annual incidenceof C_(j) is constant from age N-3 to N, and that C_(J) is equally likelyto evolve to C_(J+1) in 1, 2, or 3 years. The duration pdf from the“lead to” relation CL_(j->J+1) is therefore uniform over years 0 to 3.Consequently, C_(J) beginning at age N-3 is as likely to continue to ageN-2 as to age N-1, but will not he prevalent at age N in either case.Conversely, most cases of C_(j) beginning at age N-1 would be prevalentat age N. To accommodate the uncertainty regarding the onset time ofC_(j+1), the duration pdf is reversed in time(as in the previous step),then converted to a cumulative probability function. The highestcumulative probability occurs just before the age of presentation.

[0282] Formula 3. Reversed cumulative probability (RCP) of duration ofhealth condition C_(J):

RCP(n)=Σ(DurationPDF(CL_(j−>j+1), N-y)) y=0 to n

[0283] Where:

[0284] N=age at presentation

[0285] y=a number of years between 0 and n.

[0286] For each year from birth to the age of presentation, theincidence and reversed cumulative probability of duration are multipliedto obtain a weighting factor for the onset of C_(j) in that year, arandom weighted selection chooses the year to propose as the age ofonset for the health condition C_(j). This age represents a secondproposal for the age of onset of (P_(l)l, C_(j), E_(k)).

[0287] Formula 4. Weight (W_(n) for selecting age n for the onset ofhealth condition C_(J):

W_(n)=Incidence(C_(J), n) * RCP(n)

[0288] At this point, the simulator has completed these steps. It foundvector V₀ to have a single possible predecessor, V⁻¹. Each healthcondition listed in V⁻¹ could have been the last to develop, thereforethe simulator proposed a plausible age of onset for each. The simulatorused one of two algorithms to calculate age of onset of each condition,depending on whether or not it could identify the age at which thecondition ended.

[0289] Each proposed age corresponds to a change in one element invector V⁻¹. The collection of vectors produced by these single healthcondition changes is the set V⁻². Consequently, selecting the healthcondition to change specifies which member of the set V⁻² is part o thehistory of this simulation. Although only one vector in V⁻² will appearin the history of this simulation, all of the health conditions in V⁻¹will be traced back to birth through vectors from sets V⁻³, V⁻⁴, etc.The question is not whether each condition has a history, but whenevents occurred.

[0290] A safe strategy is to instantiate the vector from V⁻² occurringat the latest age, along with any facts that had been tentativelyproposed with that age and vector. If two or more vectors from V⁻² sharethe latest moment in age, one may be selected at random. The historygeneration step is repeated with the instantiated vector from V⁻²replacing V⁻¹ as the focus of attention.

[0291] The “lead to” relations, such as PL_(j-1->i), may need toinstantiate modifiers in order to produce a duration pdf.

[0292] Some modifiers might be defined by a history of a healthcondition in an active network. Instantiations of health conditions inactive networks create additional temporal constraints for theseconditions. These constraints typically dictate that a comorbid healthcondition, C_(x), is present at a point in time (e.g. at age N, themoment of transition from P_(i-l) to P_(l)) , for a period of time (e.g.at least five but not more than ten years), or both (e.g. for the pasttwo to four years). These conditions can be evaluated for logicalcompatibility with incidence data and the case. For instance, theinstantiation of a modifier may require that C_(x) is present at themoment of transition from P_(l-l) to P_(l). If x≢j and C_(J) is part ofthe target vector V₀, then this instantiation can not apply in thissimulation. The probability of a modifier requiring C_(x≢j) is thereforezero. A slightly different constraint indicating that C_(x) isconcurrent with P_(l-1) for five to ten years, where x=j−1, may belogically possible.

[0293] Note that the outlining algorithm will select this instantiationonly if the onset of P_(i-l) is proposed for an older age than the onsetof C_(j). The simulator can therefore be required to add C_(J) at anolder age than the onset of P_(i-l). It is important to reconcile thisage of onset of C_(j) with incidence data for C_(J), before thetentative instantiation.

[0294] The simulation algorithm does not require that exacerbationnetworks reach any particular health condition prior to changes in theirparent conditions. For instance, health condition P_(i) may permitexacerbations to reach condition E_(k), while health state P_(l-1) onlyallows exacerbations to reach condition E_(k-2). The simulationalgorithm may suggest that E_(k) developed before P_(l-1), creating anintermediate vector such as V_(l)={P_(l-1), . . . E_(k-1)}, which is inturn instantly preceded by V_(i2)={P_(i-1), . . . E_(k-2)}. Thesimulated medical history would indicate that the patient developedP_(l) and E_(k) simultaneously.

[0295]FIG. 7 is a flowchart providing an overview of the stochasticprocess. In FIG. 7, the stochastic process begins with defining a testarea or subject area to be tested in Step S2. In Step S4, the sex, race,and other genetically determined facts are assigned to the prospectivepatient. In Step S6, the past medical history of the patient isgenerated, by proposing concurrent histories for each of the healthconditions. In Step S8, the case history that will be accessible to theexaminee is generated for use in the examination.

[0296] In Step S10, the examinee or physician encounters the patient ata predetermined stage that is suitable for the examination. The examineemakes a decision as to whether treatment or intervention is appropriate,and either performs the treatment or not. The patient is optionallyevolved in Step S12 in accordance with the examinee's decision andactions performed in Step S10, and the examinee may be optionally testedagain in Step S10.

[0297] Stochastic Process History Outlining Example

[0298] Consider an examination of the management of osteoarthritis.Among several cases in this area is one describing a patient with anacute flare of osteoarthritis of the knee. The case presents withestablished grade II chronic osteoarthritis, obesity, and No GastricUlcers. No other networks are active in this case. The health conditionsin parallel networks are:

[0299] P: Grade 0 Knee Osteoarthritis (OA), Grade I Knee OA, Grade IIKnee OA, Grade III Knee OA

[0300] C: Normal weight, Obesity, Morbid Obesity

[0301] C*: No Gastric Ulcer, Grade I gastric ulcer

[0302] The health conditions Grade I Knee OA and Grade II Knee OA areassociated with exacerbation networks:

[0303]^(E)grade-II: Baseline Knee OA, Acute Flare of Knee OA

[0304]^(E)grade-I: Baseline Knee OA

[0305] The presenting vector is

[0306] V₀={P₃, E₂, C₂, C′_(l)}

[0307] ={Grade II Knee OA, Acute Flare of Knee OA, Obesity, No GastricUlcer}

[0308] The “lead to” relations required for history generation arePL_(1->2), PL_(2->3), PL_(3->4); EL_(1->2), EL_(2->1); and CL_(1->2).The “lead to” relations required for evolution are PL_(3->4); EL_(2->1);CL_(2->1), CL_(2->3), and C*L_(l->2).

[0309] The normal health condition in the Egrade_(-II) exacerbationnetwork, Baseline Knee OA, may be instantiated twice. The Acute Flare ofKnee OA health condition is incident, and all other conditions areprevalent.

[0310] Age-specific prevalence data about the presenting healthcondition in the primary network, Grade II Knee OA, conditioned on sex,race, and other essentially predetermined and generally permanentpatient characteristics are provided.

[0311] The probability of generating a white female patient, given acase of Grade II Knee OA is asserted to be 63%, the fraction of all OAcases found to occur in white females.

[0312] When sex and race are selected, the state of the prevalence nodeis defined. The prevalence node supplies the prevalence of Grade II KneeOA in white females as a shape defined by the points {(0 years, 0%); (25years, 0%); (35 years, 0.2%); (60 years, 5%); (100 years, 45%)}. Theprevalence of Grade II Knee OA at any specific age is found by linearinterpolation, so that the prevalence at age 20 is zero, and theprevalence at age 80 is 25%. The rapid rise in prevalence from age 60 to100 suggests a high probability of generating a very old patient,because these data do not reflect the scarcity of very old people.

[0313] To correctly simulate the age distribution of patients, anabsolute prevalence is calculated using formula 1. Assume that theprevalence of the special condition “Alive” for white females is aroughly sigmoid curve with a median survival around 78 years, such as{(0 years, 100%); (1 week, 99.9%); (1 year, 99.8%); (15 years, 99.5%);20 years, 99.2%); (50 years, 95%); (60 years, 85%); (80 years, 30%); (90years, 8%); (99 years, 05%); (100 years, 0%)}.

[0314] Formula 1 produces absolute prevalence weights including thepoints {(0 years, 0); (2 years, 0%); (35 years, 0.2%); (50 years, 2.9%);(60 years, 4.25%); (80 years, 7.5%); (90 years, 2.8%); (99 years, 0.2%);(100 years, 0%)}. The peak absolute prevalence (8.77%) of Grade II KneeOA therefore occurs at age 73 rather than age 100, and absoluteprevalence is skewed toward younger patients, so that the median age ofprevalent cases is 71. The product of the Alive and Grade II Knee OAprevalence is similarly multiplied by the prevalence of the obesity andNo Gastric Ulcer conditions. This could further skew the agedistribution away from the elderly as obesity, a risk factor for deathat relatively young ages, is less prevalent in older patients.

[0315] Finally, the incidence of Acute Flare of Knee OA is obtained, ifit is available. Since this health condition is part of an exacerbationnetwork, it might be safely assumed to be equally likely to occur at anyage where its parent, Grade II Knee OA, is present, if the incidence ofE_(k) is not specified. In this case, no further adjustment to theprevalence product produced above is required.

[0316] In general, the incidence shape for an incident health conditioncan be multiplied by the product of the prevalence shapes obtainedabove. One year is chosen at random from the resulting distribution in aweighted random selection process. We will assume that the processselects age 70 for this patient's presentation. This means that a whitewoman with a history of Grade II Knee OA, Obesity, and No Gastric Ulcer,presents at age 70 with an acute flare of her osteoarthritis.

[0317] The next process generates the past medical history of thepatient, by proposing concurrent histories for each of the healthconditions in the presentation vector V₀={Grade II Knee OA, Acute Flareof Knee OA, Obesity, No Gastric Ulcer}. The first step in this processtraces health condition transitions as illustrated in FIG. 8.

[0318] As illustrated in FIG. 8, the Acute Flare of Knee OA is incident,so that its precursor, Baseline Knee OA, must be present in vectorV⁻¹={Grade II Knee OA, Baseline Knee OA, Obesity, No Gastric Ulcer}. Theage of onset of V⁻¹ and the preceding vector V⁻² are obtainedsimultaneously by predicting when each element of V⁻¹ might havedeveloped, and asserting that the last predicted change did occur.

[0319] Grade II Knee OA, an element of vectors V₀ and V⁻¹, willeventually evolve to Grade III Knee OA. A history generating relation,Grade II Knee OA leads to Grade III Knee OA, describes how long thismight take, perhaps 5 to 10 years. If this relation posits a shorterinterval between these conditions, then the simulation is constrained toproduce patients with a recent onset of Grade II Knee OA. If the historygenerating relation posits a longer interval, then patients may have along established osteoarthritis condition.

[0320] Grade II Knee OA is prevalent in vector V₀, presenting at age 70,and wit no more than 10 years allowed for evolution to Grade III kneeOA, the earliest age at which the grade II condition could have appearedis 60 years. If so, this patient remained a longer time than usual inGrade II Knee OA, and the transition to Grade III Knee OA is expectedshortly. The patient is most likely to have developed Grade II Knee OAbetween age 65 and 70, among a cohort in which no one would haveprogressed to Grade III Knee OA by age 70. If the incidence of Grade IIKnee OA rises from age 60 to 70, the product of the reversed cumulativePDF and the incidence shapes will be further skewed towards later ages.We will assume that age 65 years is randomly selected from this product.

[0321] A similar procedure produces an age of onset for obesity. Ahistory generating relation, Obesity leads to Morbid Obesity, describesthe length of transitions, perhaps 10 to 25 years. Obesity is prevalentin V₀, so a reversed cumulative PDF is multiplied by the incidence ofObesity, and an onset age between 45 and 60 is proposed.

[0322] The No Gastric Ulcer element in V₀ is a stage 0 condition, whichmight evolve to stage I at some time. Since the incidence of stage 0conditions is always between 0 and 1000 at birth, but is always 0% afterbirth, so that the duration PDF is irrelevant to the selection of theage of onset, as long as the reversed cumulative duration PDF isnon-zero at birth.

[0323] Finally, the Acute Flare of Knee OA condition has a known onsettime, at age 70. The history generating relation, Baseline Knee OA leadsto Acute Flare of Knee OA, describes the duration of Baseline Knee OA,perhaps 3 to 12 months. If the duration of acute flares is very short,and there are no other conditions in the exacerbation network, then thisPDF also describes the periodicity of flares, given the presence ofGrade II Knee OA. If specific incidence data for the acute flarecondition are not available, the incidence of the parent condition forthe exacerbation network (Grade II Knee OA) can be substituted. Theproduct of the reversed (but not cumulative) duration PDF and theincidence supplies a distribution from which to select an age of onset,for instance 69 years, 7 months. Since this is the oldest age proposed,it is selected and instantiated. Step 2 of this process, illustrated inFIG. 9, is analogous to Step 1 described above, and therefore, noadditional discussion is described herein.

[0324] Finding Generation for Stochastic Process

[0325] Finding generation adds detailed descriptions of patients'features to the outline generated in the steps above. Beginning with ahealthy newborn patient (or embryo) of the specified sex and race, thefinding generation process assigns values of specific findings expectedin healthy individuals. These may change when the patient develops a newhealth condition at the age selected by the outlining process.

[0326] The patient's detailed features are generated using modelinginstructions stored as Reasoning elements with health conditions.Specific findings associated with normal health are created in asequence indicated by these instructions. Each Specific finding isinitially defined from the onset of life until age 100. For instance,the patient's height is derived from a randomly generated percentile anda set of shapes resembling a pediatric growth chart extended to age 100.The set of shapes used may be conditionally dependent on the sex, race,and any other established facts about the patient.

[0327] The finding generation process should generally create dependentfindings, e.g. knee pain, after generating the findings upon which theydepend, e.g. joint space narrowing. Careful selection of findings torepresent may reduce some dependencies. For example, the model ingeneral is more robust if height and body mass index are considered tobe independent findings, and weight is not calculated until explicitlyrequested during a simulation. Therefore, the model in general is morerobust if height and body mass index are considered to be independentfindings, and weight is not calculated until explicitly requested duringa simulation. Most findings are instantiated as a series of pairs ofvalues and ages. Values at other ages may be found by linearinterpolation.

[0328] Findings may vary with predictable circadian, lunar, and annualrhythms, described by shape subpatterns. Shape subpatterns can becombined with a shape to produce fluctuations on realistic temporalscales.

[0329] Finding distortions illustrate events having temporary effects onthe shape of some value. For instance, a temperature shape during afebrile illness might be 39° C., with a distortion pattern indicating a1° C. drop for four hours following administration of acetaminophen. Theexact temperature reported at a given time would depend on the currentvalue of the lifetime temperature shape and whether the patient consumedacetaminophen in the last four hours.

[0330] After determining patterns for all findings present at a point intime, the simulator proceeds forward in time to the next healthcondition vector. The simulator updates findings for the new situation.This loop continues until the computer has described the findings of thepatient in the final health condition vector.

[0331] Using Pre-Generated Patients

[0332] In accordance with one design of the present invention, when thecomputer based examination system generates and evolves a randompatient, it cannot reuse the patient information if the patient isevolved once. That is, every time the examination is executed, we needto generate a patient to continue the test. Not only does the process ofgenerating a patient take tremendous time, but also the evolved patientcannot generally be tested again in the future.

[0333] In accordance with another design of the invention, the patientis pre-generated, evolved and stored in the Whiteboard database. Thepresentation system can test the patient in countless time if wanted.Furthermore, different physicians can test the same patient at the sametime.

[0334]FIG. 10 is an illustration of the entity-relationship model whenpatients are not pre-generated, and FIG. 11 presents the modifiedentity-relation diagram of the modified Whiteboard database when thepatient is pre-generated. Each node represents a status of a patientwith parallel health states. For example, when a patient is generated,he or she is located at node 1, the patient might be evolved to severalstatus located at node 2, 3, 4 . . . , etc. Therefore, a patient canhave many nodes.

[0335] Many nodes can share same EXHIBITS and HAS. For instance, when apatient is evolved to a severe knee problem, we first take out the mostupdated EXHIBITS of the previous node, modify it and then write it tothe new node, and at the same time generate a new EXHIBITS for the newnode. The new node will point to the EXHIBITS prior to the most updatedEXHIBITS of the previous node. If nodes are in the same content area,they also share the same FINDINGS and PATTERNS, but their shapes aredifferent, which can be found in table Pattern_Shape.

[0336] Since different physicians can use the same patient for the testat the same time, the corresponding action contents needs to be givenfor each physician. Therefore, every time a patient has a new node, wealso generate the patient's action contents. When the physician gets tothe patient with the specific node, the action contents are copied tophysician_actions tables.

[0337] The table ACTIONS, HEALTHSTATE and ACTION_HEALTHSTATE arepre-generated, and a corresponding utility integrated withpre-generating COA is created. Accordingly, the evolution process forpre-generated patients is for example, as follows:

[0338] a) Based on the parallel health states of the patient at thespecific node, fetch all corresponding actionID from action_healthstate.

[0339] b) Based on the possible target of each actionID, construct allcombinations that lead to different parallel health states.

[0340] c) Create a new node for each possible action combination.

[0341] d) Copy the SHAPE from old node to the new node.

[0342] e) Construct a tuple in table NodeToNode where the actioncombination, old nodeID and new nodeID will be stored.

[0343] Generating Patients with Parallel Health State Networks

[0344] A detailed description of parallel health state networks is nowdescribed. We have determined that parallel health state networksprovide a model with a reasonable biological basis, more easily defineddata, greatly improved reuse potential, and a better segmentedimplementation. Evolution of synergistic health problems (e.g., viciouscycles) are managed using structures from the original data model. Aworking patient generation process is creatable using the parallelnetwork model.

[0345] We have determined that the number of conglomerate health statesexpands combinatorially, and the incidence and duration of theseconglomerate health states is often a matter of speculation or isredundant with previously stored information.

[0346] We have also determined that a parallel network approach improveson the accessibility and reusability of health state data, whileretaining the ability to handle the dependencies inherent in synergisticcycles.

[0347] Humans are composed of inter-dependent cells organized intotissues and organs. Some tissues directly or indirectly control thestate of cells in other organs through mechanical, neurohumoral, orother processes.

[0348] An individual's health reflects the current health of all ofthese cells. Therefore, a very high resolution model of the life of ahuman body might describe the histories of the cells comprising thebody, including their dependency on other cells. In clinicallyrecognizable processes, the cells comprising one tissue share similarstructure, function, and health with many of their immediate neighbors.Their health may diverge rapidly from the health of the cells in othertissues. Therefore, a model concentrating on the histories of tissuesretains considerable resolution.

[0349] Each tissue can be imagined to evolve on its own standardschedule unless some local insult occurs, or an insult to another tissuealters the schedule. The normal tissue schedules proceed in parallel.For instance, bone, Islets of Langerhans, nephrons, and retinal tissueall gain and lose function at predetermined rates. If bone losesfunction (strength), a local pathological parallel process (fracture)becomes more likely. If Islet cells lost function (insulin secretion),distant pathological parallel processes in nephrons and retinas becomemore likely or progress more rapidly (diabetic nephropathy andretinopathy).

[0350] Without parallel networks, distractors, such as randomlyappearing colds or a history of appendicitis might require manyconglomerate states. Also, information collected for one disease domainmight have to be completely replicated in other domains (for instance,obesity descriptions would occur in osteoarthritis, diabetes,hypertension, combinations of the above, and independently).

[0351] We have also determined that many therapeutic complications areacute site-specific illnesses superimposed on an antecedent illness. Onthe other hand, some problems interact in synergistic cycles:Osteoporosis increases the likelihood of fractures, and immobility(following a fracture) increases the rate of progression ofosteoporosis. Consequently, many of the most interesting diseaseprocesses are intertwined with others. In a network of conglomeratehealth states, these dependencies can be explicitly described at nodesand along edges between nodes. In a parallel network model, theinteracting networks must be aware of each other.

[0352] This view of health and function, we have determined, suggests adefinition of parallel health state networks: A parallel health statenetwork for a tissue describes a collection of clinically discoverableand mutually exclusive states in which that tissue may exist, andpossible transitions between states. For example, the normal developmentof a tissue, described from a person's birth to death, is one distinctstate in a network.

[0353] Physically separated cells of the same tissue type may exist invery different states. For instance, the left and right knee joints aresusceptible to pathologically indistinguishable osteoarthritic changes,but one knee may exhibit more advanced changes than the other.Therefore, parallel networks require identification of involved sites.

[0354] A parallel network is, not coincidentally, a disease stagingscheme. Parallel networks for chronic diseases are typicallyrestatements of familiar staging concepts (e.g., Stage O or no disease,followed by Stage I or mild disease, and so on). The parallel networkillustrates these as sequential stages, even in acute processes such asankle sprains or burns. A third degree burn is always preceded by asecond degree burn, if only for the briefest moment of time.

[0355] Parallel networks alter knowledge acquisition and storagerequirements, as well as patient generation algorithms, when compared toconglomerate health state models. Diagnoses previously combined in aconglomerate state become distinct states in different parallelnetworks. The conglomerate health state of the body is described by avector indicating the current status of all parallel networks.

[0356] Illustrations of their disease domains help medical expertsunderstand the scope of their knowledge acquisition task. Initiallyintricate domain models were decomposed into much less threateningparallel networks. FIG. 12 illustrates parallel network structures. Thesimplest network is a collection of one or more static states, typicalof genetic (Down's syndrome) and some congenital conditions(anencephaly). The progressive network is a series of states with nocycles, typical of degenerative illnesses such as osteoarthritis. Thereversible network illustrates chronic but reversible conditions, suchas essential hypertension and weight disorders. In the injury network anacute insult evolves to either recovery or a chronic condition with alater recovery. Injury networks describe many infectious diseases andtrauma.

[0357] The addiction network illustrates that a person may abstain from,use, abuse, or become addicted to something; in the current model, apreviously addicted person can only be addicted or recovering, butcannot return to abstinence, use or abuse. The surgical interventionoverlay illustrates that new states can be added to the above networksusing irreversible therapies such as radiation or surgery.

[0358] Parallel networks of three types are identified. The primarynetwork contains the diseases that define the domain, such as diabetesmellitus. The second type of network contains a risk factor forprogression through the primary network, such as obesity. The third typeof network includes complications attributed to states in the primarynetwork or its management, such as retinopathy.

[0359] We have also determined that the following information is used tocreate parallel networks: 1) how long a risk factor should exist beforeit could influence a transition between states in a primary network, 2)the time required for transitions in the primary network, givendifferent combinations of risk factors, and 3) the number of passes anindividual patient should be allowed to make through a cycle (e.g., fromacute injury to recovery back).

[0360] The data model objects originally intended to store risk factorsincluded a “Person HAS Health State” relation, which identified a healthstate, its onset and duration. In addition, HAS relations indicates apreceding HAS relation to support tracing of medical histories. Theseattributes are adapted to describe parallel synergistic networks.

[0361] The patient generation process uses a weighted random process toselect all times and events, starting with an age of onset for a healthstate on the primary network. Risk factors are selected next. Unlike theconglomerate health state patient generation algorithm, any diagnosesassociated with altered risk must be described in a parallel network.The plausible range of duration for each risk factor is stored in a HASrelation, and used in selecting its onset age. If the risk factorevolves independently of the primary network, the HAS relation does notindicate a preceding HAS, and the algorithm creates the risk factorhistory using default assumptions in its parallel network. If theprimary network does interact with the risk factor, the preceding HASrelations provide time constraints that promote plausible concurrentevolution of the primary and risk factor networks.

[0362] The original history generation algorithms are used withinindependently evolving parallel networks. Consequently, the systemcontinues to support conglomerate health states described as a parallelnetwork. In contrast to the conglomerate health state model, theparallel network technique may require explicit and separate generationof the histories of the primary network and any number of risk factors.

[0363] Computer Implemented Process

[0364] The process of the computer based examination or assessmentsystem is described in detail in connection with FIGS. 13-14. Thecomputer implemented process includes the overall concept that thephysician is presented with an examination, and the process generatesmultiple instances of patients. These generated patients representclinical scenarios that a physician would have to go through toadminister proper treatment. These scenarios are stored in a white boarddatabase which stores both the database implementation (i.e. thepatients stored in data structures), as well as computer codes whichoperate from base structures including information on physician.

[0365] There are three basic actors in the computer based examinationsystem: physician, white board and patient generator. Thephysician/examinee initiates the white board action by logging in. Oncethe examinee logs in, then the white board makes one or more requests tothe patient generator. The white board generally provides the patientsimulator with the basic testing area. The patient generator then startsthe process of generating the patient and evolving backwards, andoptionally forwards in time for pre-generated patients. Thus, thecomputer based examination system includes separate programming objectsin the general C++ programming sense for physician, the patient and thewhite board.

[0366] In general, the physician/examinee pre-registers to take theexamination, and provides (or the system already has stored) detailedbackground information on the physician, areas of weaknesses, priorexamination information, and the like. Thus, the physician logs-in tothe computer based examination system in Step P2, and the systemvalidates the physician in accordance with predetermined criteria, e.g.,user ID, password, correct examination, and the like.

[0367] The physician/examinee is either presented with an optionallist(s) of subject areas for examination or mandatory subject areas forexamination in Step P4, responsive to information stored in thewhiteboard database via requests thereto in Step P12. Alternatively, theexamination areas might be hidden, and the examinee might be told thatthis is a diabetic problem, with certain management issues. The examineemay optionally have a series of selections, whether it is in terms ofindividual patients or they could be in specific areas.

[0368] In some instances, the examinee may be provided a patient withsome specific statements about the patient. The computer implementedprocess may optionally determine whether the physician has been examinedbefore. If the answer is yes, then the physician might require, forexample, five of fifteen specific subject areas for the examination, ofwhich one or more would be available for testing.

[0369] In addition, prior performance of the physician may also beconsidered using a pre-stored or generated physician profile via StepP6, and requests to the prior physician performance via Step P8. Thespecific exam content is then requested in Step P10 responsive to atleast one of physician profile, prior performance, content areas.Accordingly, one or more of prior performance, the physician profile,the content of the examination, are used to provide a selection list ofthe physician to choose from in Step P14.

[0370] Depending on the above information, the patient generator processis then initiated to create a patient for the examination in Step P16.The patient generation process may be performed in Step P18 in real-timefor each patient, or may be pre-generated as described above. Under thereal-time scenario, the selection of a problem area in Step P14translates into a target health state or area.

[0371] For example, if the problem area selected was diabetes, thetarget health state in the knowledge base would be diabetes. Using thetarget health state, there are generally a plurality of health statesassociated therewith. The computer implemented process then optionallyrandomly selects one of these areas as a precursor health state in StepP20. For example, a mild case of diabetes may be the precursor healthstate for normal health state of diabetes.

[0372] The selection of the precursor health state is based on, orcalculates, onset age in Step P22 via incidence data in Step P24. Thehistory generation computer process is a mechanism that sets up areasonable beginning time and ending time for the patient that is beingpresented. The computer process chooses a target health state, precursorinformation, sex and race from the target health state, and establishesthe age of the patient. The computer process then moves backwards intime to establish onset age when the condition occurs, and proceedsbackwards in time all the way to the normal state. Next, the processmoves forward in time to determine potential subsequent health statesfor the patient based on a variety of possible interventions performedby the examinee. Thus, the process has two stages.

[0373] Depending on the precursor information/health state, informationsuch as the sex and race, along with disease prevalence in Step P34,mortality data Step P36 and incidence information in Step P24, are usedto select the specific sex and race for the simulation.

[0374] The mortality data is based on sex and race. The sex and race isselected from the health state, incidence or prevalence data and sex andrace specify mortality data. For example, if the health problem that ispresented to the examinee is new to the patient, then it is incident(e.g., a recently broken bone). Alternatively, if the health problem isan old established problem such as long term diabetes, the healthproblem is prevalent. Thus, the incidence and prevalence is insertedinto the patient case history over and over again depending on theparticular problem. Accordingly, a pre-determined decision is generallymade as to what types of disease are to be tested, prevalent disease orincident disease.

[0375] The sex and race selection uses the disease prevalence andmortality data in the sex and race selection process. The mortality dataand disease prevalence are used to establish the reasonable ages andsexes, and also ages of on-set. For example, this mechanism preventsgeneration of pregnant males, 14 year old Type-II diabetics, and thelike.

[0376] For example, the computer system determines, or is instructed asdescribed previously, that the problem area is arthritis. The sex, race,and age of the patient are determined, for example, at the point in timewhere treatment may be necessary. The patient history is then generatedback through the process/time to establish onset times of the variousdifferent health states. That is, from, for example, the point in timewhere the arthritis is severe, the patient history is generated at apoint when the arthritis was mild, and back to when the arthritis wassubstantially normal.

[0377] When the sex and race selection process is completed via thecombination of sex and race selection in Step P38 and onset agecalculation in Step P40, a patient has been generated at a specificpoint in time with a specific health state problem and thecharacteristics of that problem. Thus, the computer process hasgenerated the patient, moved backwards in time from the disease onsetage all the way back to normal. For example, if the computer processstarted with a mild condition for a specific disease, the computerprocess goes backward one time interval to normal from mild. If thecomputer process begins with moderate, the computer process will movebackward in time from moderate.

[0378] As a result of the computer process, a patient template is alsogenerated in Step P26 using the onset age determination in P40 and sexand race selection in Step P38. In addition, the generated patient isgiven a name in Step P28, and age including a date of birth in Step P30.The physician/examinee is then provided with the history of the patientfor use in diagnosing or prescribing treatment for the generatedpatient. The patient history includes, for example, age of the patient,race and sex. Up to this point in the computer process, the patient iscreated. From this point of the examination/computer process andforward, the patient and physician's interaction with that patientdetermine both the information provided to the physician/examinee, aswell as potential evolution of the patient. Changes in the patient'scharacteristics is a function of physician's action or inactions usingthe evolutionary process described below.

[0379] The evolutionary process is performed using the knowledge basestructure or entity relationship model described in detail above. Theknowledge base structure has been separated from the white boardstructures described above for administrative purposes, butalternatively may also be combined therein. The knowledge baserepresents all the information that does not necessarily have to go withthe patient for purposes of presentation to the examinee. The knowledgebase includes information used to create the patient and provideinstances of information.

[0380] However, separating the knowledge base from the white boardstructure has the advantage that the computer generated patients do notrequire as much data to be transported therewith. Accordingly, aseparate structure is created called the white board structure. Thewhite board structure advantageously includes the information requiredto generate the patients and to present the patients to thephysician/examinee. The white board structure includes informationcontaining patient description and all the findings that are typicallygenerated that are not necessarily related to the problem. For example,blood pressure, blood glucose, and the like.

[0381] That is, the white board structure provides all information thatis generally available to the examinee, such as information satisfyingexaminee queries on prior history, laboratory tests, and the like. Inaddition, when pre-generated patients are used, all findings associatedwith the patient including all pre-generated evolutionary states arealso stored in the white board data structure.

[0382] For example, if the patient had moderate arthritis, the patientmay generally transition to two other health states: severe arthritis,or mild arthritis. Thus, in one embodiment of the invention, thecomputer process pre-generates the possible health states for thepatient. According to this embodiment where the patient ispre-generated, the process of evolving a patient may, in somecircumstances, be more computationally efficient than to generate thepatients dynamically. Thus, for pre-generated patients described abovein detail, all possible states are generated ahead of time and then usedby the white board structure in accordance with the pre-generated statewhen activated or selected by the examinee.

[0383] The white board accesses the patient template in Step P42, andgenerates the patient record in Step P46, responsive to requestsinitiated by the white board to the patient history information in StepP44. The patient record is not generally reviewable by the examinee,except on individual requests by the examinee in Step P48. The examineerequests information from the patient record in Step 48 which providesthe examinee the physical view of the patient. For example, thepatient's blood pressure may be stored in the patient record forretrieval by the examinee. Other examples of information stored in thepatient record include chief complaint, past medical history, pastpatient behavior or compliance information.

[0384] The white board will also generate examinee actions and patientinterventions in Step P52 by reviewing and evaluating the physicianintervention in Step P50, responsive to the patient record. The examineeactions and patient interventions contribute to the patient evolutionconditions used in the patient evolution process described above indetail.

[0385] Whether the patient is pre-generated or not, the computerprocess/patient generator generates the initial patient, andsubsequently evolves the patient, and subsequently presents same to theexaminee. The patient is generated by the patient generator accessingthe patient evolution conditions in Step P54, the target health state inStep P56, and any existing parallel health states in Step P58. Thepatient is evolved by the patient generator in Step P60 to the evolvedhealth state, which may become the target health state in Step P62.

[0386] At this point we have a patient on the white board presented witha particular health state, which typically is the form of a chiefcomplaint. From this time on, the examinee/physician takes control ofthe process, and nothing is going to happen in the computer basedexamination system unless the examinee/physician does something, unlessthe health state is time dependent and able to advance to another stateautomatically, such as by inaction on the part of the examinee.

[0387] For example, if the health state is an acute problem, such as aheart attack, there may be a time dependency built in that is going toforce some action of the physician within a specific time before thepatient experiences another heart attack. In this example, the examineeusing the computer based examination system may dismiss the patient, thepatient will walk out of the doctor's office/hospital, and the examineewould receive notification that the patient just showed up in theemergency room with a problem.

[0388] Alternatively, if the examine is too slow in diagnosing anillness, the inability to treat the patient in a short period of timemay also result in the patient progressing to a different health state.For example, a patient that has a heart attack might progress to a moreserious state if the examinee does not perform corrective measures veryquickly while the patient is, for example, in the hospital. In general,allowing time to elapse without intervention is an intervention choicealong with the other active interventions that an examinee might choose.

[0389] In order to determine the target health state, the “iterate untilnormal reached” process is initiated via Step P64 which sets one or morepre-cursor health states to the target health state. The “iterate untilnormal reached” process iterates in Step P66 until the normal healthstate is reached backward from the target health state. For example, ifa mild health state is selected, the precursor health state is normal.The “iterate until normal reached” process also establishes one or moreoptional parallel health states in Step P68. Precursor parallel healthstates are hen generated as needed in Step P70, which are then used tocontribute to the patient history in Step P72.

[0390] The computer based examination system ensures that the age ofonset for the various parallel health states is reasonable. Thus, theprocess of generating precursor health states for the parallel healthstate is a multi-dimensional process of monitoring health states to beconsistent, to prevent unreasonable scenarios, time frames, and thelike. If the parallel health states are related, they have to be relatedto each other sufficiently enough so that the evolution of health statesmakes sense.

[0391] The parallel health states are also used to establish thefindings in Step P74, which contribute to the patient history in StepP76. While the above steps have been described in, more or less, asequential manner, it should be clear that the various steps describedherein may be performed in parallel, independently, and/ornon-sequentially, as needed or for computational efficiency.

[0392] Advantageously, the computer implemented process includes thecapability of utilizing parallel health states as part of the patientgeneration process, which is described above in detail. As part of thegeneration process, a decision is generally made to include or excludethose particular parallel (e.g., morbid or co-morbid) health statesalong with the original state of the disease.

[0393] We have determined that sometimes health problems tend to beconcurrent, but they are not generally defined as being necessarilyinterrelated. The computer based examination system provides the featureof handling a plurality of health states, either related or not relatedto each other. For example, there tend to be lots of people withdiabetes and high blood pressure. Accordingly, we define these twohealth states as related to each other. Alternatively, the plurality ofhealth states may be considered to be substantially independent andstill within the scope of the computer based examination system of thepresent invention.

[0394] The present invention further provides the feature of dealingwith parallel health states substantially or completely independent ofeach other to permit dependent or independent management decisions. Forexample, a person that has diabetes and high blood pressure generallyrequires slightly different management decisions than a person who justhas high blood pressure or a person who just has diabetes. For example,the physician/examinee may prescribe a more expensive anti-hypertensivedrug if the patient has both diabetes and high blood pressure because ofpotential complications unique to the combination of health states.Thus, the computer based examination system may be used to determinewhether the examinee has made the appropriate management decision.Alternatively, the computer based system may be used to collect variousresponses from different well recognized physicians to establish aminimum level of care for insurance companies, health careorganizations, other physicians, and the like.

[0395] The present invention also provides the feature of providingdistractions when attempting to diagnose the disease/illness. That is,the patient may include symptoms and/or indications that might berelated to the problem seemingly presented to the examinee, but, infact, these indications distract the examinee along an inappropriatepath, such as excessive testing, over-prescribing medications, and thelike. Accordingly, we have also determined that distraction makes a goodargument for having parallel problems.

[0396] At this point and time, the computer based examination systemselects an area for examination, and is in the process of workingbackwards in time. The process iterates from precursor health state downto the normal health state, where at each precursor state the processconsiders potential co-morbid problems. Both the precursor or subsequenthealth states are the primary problem, and the parallel health statesgenerate findings. The findings are a part of the patient history. Forexample, a finding of obesity might be a change in weight. The processmoves backwards while at the same time looking at potential parallelhealth states have been substantiated. The history of findings aregenerated at the white board level.

[0397] Now if the physician takes some action at this point in time thatcauses patient evolution (that is, the physician causes some actionwhich the white board is checking at this point and time), the whiteboard matches the action up against something that is going to causepatient evolutionary health state change. The white board then makes arequest to the patient generator to evolve the health state.

[0398] If the full patient has been generated on the white board, thenthe patient generator is replaced with the white board itself to providea pre-evolved patient from memory. If, however, the knowledge base islinked for a dynamic situation, then the patient generator dynamicallyevolves the patient. In either situation, the evolved health statebecomes the target health state at this time. For example, the healthstate has evolved from mild to moderate arthritis, or from mild obesityto moderate obesity.

[0399] The computer implemented process also includes the possibility oftreating patients with management health issues that do not generallybecome totally normal (e.g., long term diabetes, arthritis, and thelike), as well as health conditions that may return to completely normal(e.g., broken bone, and the like).

[0400] In fact, we have determined that it is particularly likely thatpatients will revert to normal conditions when the patient experiencesan exacerbation health state/condition for the computer basedexamination system. For example, we have determined that an exacerbationcondition can have, for example, mild, moderate and severe states. Ifthe patient has a moderate exacerbation, there is a chance that thepatient experienced a mild exacerbation before evolving to the moderatestate. There is also a chance that the patient had a severeexacerbation, is now recovering, and may return to the normal state afew minutes later.

[0401] In summary, the computer based examination system utilizes threeactors, the white board the physician and the patient generator. Theexaminee initiates the whole process by logging in. The examinee logsinto the white board, and the white board accesses various informationto determine whether the examinee is valid. If the examinee is valid orverified, the white board looks up the examinee's profile to determineany background specifics on the examinee, such as specific areas needingimprovement, past examination results, and the like.

[0402] The white board then determines or is provided the exam content,and then contacts the patient generator. The patient generator beginsthe generation process, selects the disease or subject area, andcontrols the actual combinations of health states and co-morbid healthstates via a case structure. The case structure controls both thepresenting health state as well as the co-morbid health state. The casestructure filters the generation process and makes a predetermination toeliminate predetermined impossible situations, or difficult orunimportant situations that are not to be used in the testing. The casestructure indicates that even though a specific health state or parallelhealth state is in the knowledge base and even potentially legitimate,the case structure will not present that problem. Thus, the casestructure simply controls which of the health states will be presentedto the examinee, and which of the co-morbid health states, and possiblyflare states will also be presented to the examinee simultaneously orsequentially.

[0403] The white board then retrieves the patient template including,for example, the patient history, the chief complaint, the assessmenttest, and the like. From this time on, the examinee performs some actionby either requesting data which is controlled by the white board or bycausing, directly or indirectly, some action to take place. Once anaction is performed, the patient may be evolved to the next health stateby the patient evolution process.

[0404] Both the request of information and the review and evaluation ofthe examinee's actions or intervention are generally handled by thewhite board for convenience, but multiple control mechanisms may also beused. If the white board sees there has been a change in health statefor the patient, then the white board would then go to patient evolutionprocess to initiate the evolution, and request the patient generator toprovide information regarding the evolved patient.

[0405] The patient evolution information may optionally be pre-generatedfor computational efficiency. That is, even when patients are createddynamically, some predetermined evolution information may be ready foruse by the computer based examination system based on the potential ofthe possible evolution periods/health states. For example, if the targethealth state was moderate, the computer based examination system mayhave predetermined onset time of moderate, and therefore, know the timefor mild, normal, severe, and the like. In effect, the white boarddatabase/object optionally includes a complete possible look at thefuture appearance, as well as the past characteristics for thisparticular patient at a particular health state.

[0406] The underlying goal of the computer based examination system isthat the evolutionary process is generally the same as the patientgeneration process. Both processes are generally the same, just thegeneration process has more steps to generate the patient. In theevolutionary situation, the computer based examination system deals withmultiple possible health state successors in different parallelnetworks. The zero state, or state where the examination begins,generally has a primary health state like moderate arthritis, possibly aflare state such as an acute swelling in the knee, and comorbid statessuch as overweight.

[0407] To generate the patient history, the computer process take themoderate arthritis, the flared up knee, and the overweight condition andlooks backwards in time to determine the most recent precursor state.For example, the precursor of the moderate arthritis could be mild, theprecursors of the flare could be baseline or normal, and the precursorfor the overweight condition could be normal weight. The computerprocess sets the patient's current age, for example, as age 50, and nowmoves backward in time.

[0408] For example, for a 50 year old person with moderate arthritis, itis likely that the arthritis began 5-10 years ago. With respect to theflared knee, it is likely that this condition began within the lastcouple of days. For the overweight condition, it is likely that the 50year old person began this condition 10-15 years ago. Therefore, thereis a 5-10 year interval (arthritis), a 3 day interval (flared knee), anda 10-15 year interval (overweight).

[0409] The computer process moves backward in time to that last changethat should have occurred. In this situation, the first precursor healthstate is the flared knee which occurred 3 days ago. The clock then getsreset, and the next earliest precursor health state is determined. Thewhiteboard generally throws away all the previous information that wasused to generate the later precursor health state, and recalculates thenext earliest precursor health state until all precursor health statesare generated to the normal condition for all conditions/diseases.

[0410] Switching to the forward version, the patient evolution process,the computer process looks forward in time instead of backwards in time.Therefore, considering the above example, there may be a change in 3days (knee flare), another change in 5-10 years (arthritis), and anotherchange in 10-15 years (overweight). The next change that will occur willbe in 3 days. That is the evolutionary process which, similar to thepatient history generation process, recalculates onset times for eachsubsequent health state. Thus, the main difference between the historygeneration process and the patient evolution process is the data beingapplied to each process.

[0411] The computer based examination system may also be used todetermine whether specific physicians are practicing cost effectivemedicine for use by, for example, insurance companies. The system canprovide objective criteria for treating patients by defining episodes ofcare for isolated problems. For example, the computer system canindicate approximately the amount of money to spend on a patient with aheart attack with no other concurrent problems, for an asthmatic patientper year, and the like.

[0412] The computer based examination system and process provides a“flight simulator” where the physician can practice specific preferredforms of treatment, as appropriate. For example, if the patient has aheart attack, the examinee/physician should generally prescribe aspirinfor long term usage, but many do not. Thus, the computer basedexamination system may also be used as a training system so that theexaminees rehearse a desirable behavior such as prescribing aspirinafter heart attacks. The computer based examination system can thereforealso be used to increase desirable behavior when the physician interactswith a real patient.

[0413] Consequently, if a particular physician or group of physiciansare determined to be expensive for an insurance company or health caregroup, and the computer based system shows that the physicians arelikely to provide appropriate care, then it is possible that thisphysician or group of physicians have a particularly expensive patientpopulation, and should therefore not be faulted. Further, the computerbased examination system may also be adapted to receive the specificdata collected by the physician and interventions associated therewith,to further verify that the practice is delivering the appropriateservices. Thus, the computer based examination system, may be used todetermine whether a particular practice is delivering services withinpredetermined guidelines.

[0414]FIG. 15 is an illustration of a main central processing unit forimplementing the computer processing in accordance with a computerimplemented embodiment of the present invention. The proceduresdescribed above may be presented in terms of program procedures executedon, for example, a computer or network of computers.

[0415] Viewed externally in FIG. 15, a computer system designated byreference numeral 40 has a central processing unit 42 having disk drives44 and 46. Disk drive indications 44 and 46 are merely symbolic of anumber of disk drives which might be accommodated by the computersystem. Typically these would include a floppy disk drive such as 44, ahard disk drive (not shown externally) and a CD ROM indicated by slot46. The number and type of drives varies, typically with differentcomputer configurations. Disk drives 44 and 46 are in fact optional, andfor space considerations, may easily be omitted from the computer systemused in conjunction with the process/apparatus described herein.

[0416] The computer also has an optional display 48 upon whichinformation is displayed. In some situations, a keyboard 50 and a mouse52 may be provided as input devices to interface with the centralprocessing unit 42. Then again, for enhanced portability, the keyboard50 may be either a limited function keyboard or omitted in its entirety.In addition, mouse 52 may be a touch pad control device, or a track balldevice, or even omitted in its entirety as well. In addition, thecomputer system also optionally includes at least one infraredtransmitter 76 and/or infrared receiver 78 for either transmittingand/or receiving infrared signals, as described below.

[0417]FIG. 16 illustrates a block diagram of the internal hardware ofthe computer of FIG. 15. A bus 56 serves as the main information highwayinterconnecting the other components of the computer. CPU 58 is thecentral processing unit of the system, performing calculations and logicoperations required to execute a program. Read only memory (ROM) 60 andrandom access memory (RAM) 62 constitute the main memory of thecomputer. Disk controller 64 interfaces one or more disk drives to thesystem bus 56. These disk drives may be floppy disk drives such as 70,or CD ROM or DVD (digital video disks) drive such as 66, or internal orexternal hard drives 68. As indicated previously, these various diskdrives and disk controllers are optional devices.

[0418] A display interface 72 interfaces display 48 and permitsinformation from the bus 56 to be displayed on the display 48. Again asindicated, display 48 is also an optional accessory. For example,display 48 could be substituted or omitted. Communication with externaldevices, for example, the components of the apparatus described herein,occurs utilizing communication port 74. For example, optical fibersand/or electrical cables and/or conductors and/or optical communication(e.g., infrared, and the like) and/or wireless communication (e.g.,radio frequency (RF), and the like) can be used as the transport mediumbetween the external devices and communication port 74.

[0419] In addition to the standard components of the computer, thecomputer also optionally includes at least one of infrared transmitter76 or infrared receiver 78. Infrared transmitter 76 is utilized when thecomputer system is used in conjunction with one or more of theprocessing components/stations that transmits/receives data via infraredsignal transmission.

[0420]FIG. 17 is a block diagram of the internal hardware of thecomputer of FIG. 15 in accordance with a second embodiment. In FIG. 17,instead of utilizing an infrared transmitter or infrared receiver, thecomputer system uses at least one of a low power radio transmitter 80and/or a low power radio receiver 82. The low power radio transmitter 80transmits the signal for reception by components of the process, andreceives signals from the components via the low power radio receiver82. The low power radio transmitter and/or receiver 80, 82 are standarddevices in industry.

[0421]FIG. 18 is an illustration of an exemplary memory medium which canbe used with disk drives illustrated in FIGS. 15-17. Typically, memorymedia such as floppy disks, or a CD ROM, or a digital video disk willcontain, for example, a multi-byte locale for a single byte language andthe program information for controlling the computer to enable thecomputer to perform the functions described herein. Alternatively, ROM60 and/or RAM 62 illustrated in FIGS. 16-17 can also be used to storethe program information that is used to instruct the central processingunit 58 to perform the operations associated with the process.

[0422] Although processing system 40 is illustrated having a singleprocessor, a single hard disk drive and a single local memory,processing system 40 may suitably be equipped with any multitude orcombination of processors or storage devices. Processing system 40 may,in point of fact, be replaced by, or combined with, any suitableprocessing system operative in accordance with the principles of thepresent invention, including sophisticated calculators, and hand-held,laptop/notebook, mini, mainframe and super computers, as well asprocessing system network combinations of the same.

[0423] Conventional processing system architecture is more fullydiscussed in Computer Organization and Architecture, by WilliamStallings, MacMillam Publishing Co. (3rd ed. 1993); conventionalprocessing system network design is more fully discussed in Data NetworkDesign, by Darren L. Spohn, McGraw-Hill, Inc. (1993), and conventionaldata communications is more fully discussed in Data CommunicationsPrinciples, by R. D. Gitlin, J. F. Hayes and S. B. Weinstain, PlenumPress (1992) and in The Irwin Handbook of Telecommunications, by JamesHarry Green, Irwin Professional Publishing (2nd ed. 1992). Each of theforegoing publications is incorporated herein by reference.

[0424] Alternatively, the hardware configuration may be arrangedaccording to the multiple instruction multiple data (MIMD)multiprocessor format for additional computing efficiency. The detailsof this form of computer architecture are disclosed in greater detailin, for example, U.S. Pat. No. 5,163,131; Boxer, A., Where Buses CannotGo, IEEE Spectrum, February 1995, pp. 41-45; and Barroso, L. A. et al.,RPM: A Rapid Prototyping Engine for Multiprocessor Systems, IEEEComputer February 1995, pp. 26-34, all of which are incorporated hereinby reference.

[0425] In alternate preferred embodiments, the above-identifiedprocessor, and in particular microprocessing circuit 58, may be replacedby or combined with any other suitable processing circuits, includingprogrammable logic devices, such as PALs (programmable array logic) andPLAs (programmable logic arrays). DSPs (digital signal processors),FPGAs (field programmable gate arrays), ASICs (application specificintegrated circuits), VLSIs (very large scale integrated circuits) orthe like.

[0426] The many features and advantages of the invention are apparentfrom the detailed specification, and thus, it is intended by theappended claims to cover all such features and advantages of theinvention which fall within the true spirit and scope of the invention.Further, since numerous modifications and variations will readily occurto those skilled in the art, it is not desired to limit the invention tothe exact construction and operation illustrated and described, andaccordingly, all suitable modifications and equivalents may be resortedto, falling within the scope of the invention.

[0427] For example, while the above discussion has separated the variousfunctions into separate functionality, the functions may be combined,physically and/or logically, and various functions may be combinedtogether. While combining various functions may make implementationdetails more cumbersome, nevertheless, the functions described hereinmay still be accomplished to advantageously provide some or all of thebenefits of the invention described herein.

[0428] As an additional example, the foregoing discussion focusedexclusively on medical applications of the current invention.Advantageously, the invention applies equally well to creatingsimulations of other complex systems, particularly complex systems inwhich an empiric description is easier to obtain than a comprehensivemathematical description. The concepts in the invention correspond togeneric concepts that apply to complex systems in general. The labels inthe current invention and the generic concept are listed in the tablebelow.

[0429] The Population (or Person or Simulated Patient) conceptrepresents any complex system. Consider a nuclear power plant. Allbreeder reactors form a population of breeder reactors, and eachindividual breeder reactor is an independent complex system within thatpopulation. The Record concept again reflects the knowledge of thesystem held by either people or computers. The breeder reactor may haveits own Record of itself stored in a computer that supervises itsoperations. The public media and the Department of Energy will maintainother Records regarding the plant. Any of these records may containinaccuracies.

[0430] The Health State concept corresponds to a generic System State.As with Health States, the System States often apply to specific partsof the plant. The Body Site concept corresponds to a generic PhysicalComponent, such as the plant (like the body), the core (like the heart),and pipes (like the throat, blood vessels, or urethra). Each SystemState will apply to some of these components. For instance, it may bereasonable to describe the integrity of any pipe by naming its SystemState from a group of Pipe leak states. Obviously, one pipe may beleaking or ruptured while another pipe is intact, exactly as we havefound with Health States occuring at Body Parts. A System State mightinclude an error in a supervising computer's code, leading the complexsystem to respond inappropriately in some situation. This roughlycorresponds to mental illness manifested by maladaptive behavior.

[0431] The Lead to relation again connects System States into parallelnetworks. Lead to relations again contain Modifiers which describeevents that make transitions between System States more or less rapid.For instance, an earthquake might cause a fatigued pipe to twist, leak,and finally rupture, just as a sports injury can cause an ankle tendonto stretch, tear, and finally rupture.

[0432] Findings again represent observable facts about the ComplexSystem, such as the temperature of a reactor's core, the water level ofthe core, or the flow rate of water through a pipe. System States willbe defined primarily by the Specific Findings present. The exactFindings required will be provided by a generation method, such as aBayesian network that reproduces experts logic about the clusters ofFindings required to classify a Physical Component of a Complex Systemas existing in a particular System State. The simulation program assertsthat the System State required for the simulation is present, thensolves for all unkown nodes in this Bayesian network.

[0433] Courses of action again represent activities by humans, anotherexternal system, or the system itself. Generally, these will be effortsto restore or maintain equilibrium of the system, or to intentionallyprepare the sytems for a change of State. For instance, preparing thebreeder reactor for a scheduled shutdown and maintenance is a course ofaction similar to preparing a patient for surgery. Agents againrepresent inputs to the system that influence its Findings orprogression, such as cooling rods, water, fuel, or repairs to computercode.

[0434] Thus, we believe that the current invention has broadapplicability beyond the domain of medical simulations. It is especiallylikely to be useful when the behavior of a system is so complex that anunderstanding of the system defies mathematical desccription. Forinstance, this invention is not well suited to simulating the flight ofan airplane, which is fully described by physical laws. However, itmight be excellent for simulating maintenance of the airplane, which islikely to reflect obscure design decisions and even unknown, butempirically observed, interactions between design decisions. Label inthis Nuclear power plant invention Generic concept example PopulationComplex system Nuclear power plant Record Record Press releases, DOEdocumentation Health State System State Overheated core, Leaking pipeBody site Physical comp. Plant, Core, Pipe Lead To Lead to Intact pipeleads to Leaking pipe Modifier Modifier Bayesian network describing howage and Earthquake modify the pipe lead to Finding Finding Coretemperature, Water level Gener. method Generation method Bayesiannetwork describing an intact nuclear plant Course of Act. Course ofAction Manual shutdown, automated shutdown Agent Agent Carbon rod,Water, Uranium, Earthquake

[0435] Further, as indicated herein, the present invention may beapplied across a broad range of programming languages that utilizesimilar concepts as described herein. The present invention may also beused in a distributed environment/architecture, optionally using thinclient technology.

What is claimed is:
 1. A computer implemented simulation and evaluationmethod for simulating interventions including active and passiveintervention to a complex system, such as a patient having a set ofnormal and abnormal conditions such as a health state, by a user, andfor evaluating the interventions responsive to predetermined criteriaand the interventions, comprising the steps of: (a) accessing thecomputer implemented simulation and evaluation method by the user; (b)defining a test area to evaluate the user by the computer implementedsimulation and evaluation method responsive to at least one ofpredetermined criteria and a user profile; (c) selecting geneticinformation of the patient responsive to the test area; (d) generating apatient history responsive to the test area and the genetic information;(e) receiving at least one intervention input by the user; and (f)evaluating the user responsive to the at least one intervention input bythe user and the predetermined criteria.
 2. A computer implementedsimulation and evaluation method according to claim 1 , furthercomprising the steps of: (g) evolving the patient responsive to the atleast one intervention, the genetic information and the patient historyto at least one subsequent health state; and (h) evaluating the userresponsive to the at least one intervention input by the user, the atleast one subsequent health state, and the predetermined criteria.
 3. Acomputer implemented simulation and evaluation method according to claim1 , further comprising the steps of: (g) evolving the patient responsiveto the at least one intervention, the genetic information and thepatient history to at least one subsequent health state; and (h)receiving at least one other intervention input by the user; and (i)evaluating the user responsive to at least one of the at least oneintervention input by the user, the at least one other interventioninput by the user, the at least one subsequent health state, and thepredetermined criteria.
 4. A computer implemented simulation andevaluation method according to claim 1 , further comprising the stepsof: (g) evolving the patient responsive to the at least oneintervention, the genetic information and the patient history to atleast one subsequent health state; (h) receiving at least one otherintervention input by the user; (i) evolving the patient responsive tothe at least one intervention, the genetic information and the patienthistory to at least one other subsequent health state; and (j)evaluating the user responsive to at least one of the at least oneintervention input by the user, the at least one other interventioninput by the user, the at least one subsequent health state, the atleast one other subsequent health state, and the predetermined criteria.5. A computer implemented simulation and evaluation method according toclaim 1 , wherein said generating step (d) further comprises the step ofgenerating the patient history responsive to the test area, the geneticinformation, and an entity relationship model.
 6. A computer implementedsimulation and evaluation method according to claim 5 , wherein theentity relationship model comprises population, record, agents ofchange, health states, findings and courses of action.
 7. A computerimplemented simulation and evaluation method according to claim 6 ,wherein the findings include specific findings, patterns andsub-patterns describing patient behaviors and characteristics.
 8. Acomputer implemented simulation and evaluation method according to claim7 , wherein the patterns describe one or more features over time.
 9. Acomputer implemented simulation and evaluation method according to claim7 , wherein the sub-patterns describe consequences of patient relatedevents.
 10. A computer implemented simulation and evaluation methodaccording to claim 7 , wherein the patterns model time and characterizeinterrelated medical observations.
 11. A computer implemented simulationand evaluation method according to claim 7 , further comprising the stepof performing a differential diagnosis responsive to the findings, thepatterns and the sub-patterns.
 12. A computer implemented simulation andevaluation method according to claim 7 , wherein confidence in apresence of the patterns increases with passage of time.
 13. A computerimplemented simulation and evaluation method according to claim 6 ,wherein the courses of action describe tasks and methods used to apply,modify, and evaluate health state information and characteristicsdescribed in the entity relationship model.
 14. A computer implementedsimulation and evaluation method according to claim 6 , wherein thecourses of action describe patient activities, including at least one ofmedical and non-medical activities.
 15. A computer implementedsimulation and evaluation method according to claim 6 , wherein thecourses of action describe potential interventions input by the userincluding at least one of diagnostic and management strategies.
 16. Acomputer implemented simulation and evaluation method according to claim6 , wherein the courses of action comprise one or more elementarycourses of action used in to construct at least one course of action,one or more types of elementary courses of action corresponding to theone or more elementary course of action, and weighting factorscorresponding to the one or more elementary courses of action.
 17. Acomputer implemented simulation and evaluation method according to claim5 , wherein the entity relationship model includes entity relations. 18.A computer implemented simulation and evaluation method according toclaim 17 , further comprising the step of evolving the patientresponsive to the at least one intervention, the genetic information,the entity relations and the patient history to at least one subsequenthealth state.
 19. A computer implemented simulation and evaluationmethod according to claim 5 , wherein the entity relationship modelincludes a health states leads to health states relation describingpatient evolution.
 20. A computer implemented simulation and evaluationmethod according to claim 5 , wherein the entity relationship modelincludes one or more of the following relations between entities:Population Contacts Population Population Related to PopulationPopulation Interacts with Courses of Action Population Exposed to Agentsof Change Population Has Health States Population Exhibits FindingsAgents of Change Cause Health States Health States Lead to Health StatesFindings Associated with Health States Findings Link to Findings Courseof Action use Agents of Change Courses of Action Identify Agents ofChange Courses of Action Treat Health States Courses of Action AlterFindings Courses of Action Reveal Findings Courses of Action EvaluateFindings.
 21. A computer implemented simulation and evaluation methodaccording to claim 6 , wherein the entity relationship model links thefindings with the patterns to a health state, rather than linking arange of finding values to the health state.
 22. A computer implementedsimulation and evaluation method according to claim 6 , wherein thepatterns include sensitivity and specificity represented as agedependent, rather than as constants.
 23. A computer implementedsimulation and evaluation method according to claim 1 , wherein saidgenerating patient history step (d) is executed once for each simulationto generate the patient history used in said computer implementedsimulation and evaluation method.
 24. A computer implemented simulationand evaluation method according to claim 2 , further comprising the stepof repeating said evolving step (g), and said receiving step (h) aplurality of times.
 25. A computer implemented simulation and evaluationmethod according to claim 1 , wherein said generating step (d) generatesthe patient history comprising a progression of health states and riskfactors traversed by the patient from a normal health condition to aspecified health condition.
 26. A computer implemented simulation andevaluation method according to claim 1 , wherein said generating step(d) iteratively generates the patient history backwards in time from aspecified health condition to a normal health condition includingsuccessive precursor health states and onset times therebetween.
 27. Acomputer implemented simulation and evaluation method according to claim1 , wherein said generating step (d) generates the patient history usinga Monte Carlo process to multiple stochastic trees to generate aplurality of potential patient histories to be used in said computerimplemented simulation and evaluation method.
 28. A computer implementedsimulation and evaluation method according to claim 5 , wherein theentity relationship model utilizes tree structures to describe aprobability density function conditioned on comorbidities, treatments,risk factors, and the interventions.
 29. A computer implementedsimulation and evaluation method according to claim 5 , wherein theentity relationship model includes diagnostic complexities and diseaseinteraction.
 30. A computer implemented simulation and evaluation methodaccording to claim 5 , wherein the entity relationship model includesparallel networks of health states to avoid combinatoric health stateexplosion.
 31. A computer implemented simulation and evaluation methodaccording to claim 30 , wherein the parallel networks of health statesdescribe at least one of a chronic condition and non-chronic condition.32. A computer implemented simulation and evaluation method according toclaim 30 , wherein the non-chronic condition includes acuteexacerbations describing acute flares of illness that occur during amore chronic health condition.
 33. A computer implemented simulation andevaluation method according to claim 30 , wherein the parallel networksof health states form at least one of the following interactions: (1)independent interaction between the parallel networks so that patientevolution between first and second parallel networks are unrelated toeach other; (2) unilateral interaction between the parallel networks sothat patient evolution on a first parallel network is unrelated to thepatient evolution on a second parallel network, and patient evolution onthe second parallel network is related to the patient evolution on thefirst parallel network; and (3) mutually dependent interaction betweenthe parallel networks so that patient evolution between the first andsecond parallel networks are related to each other.
 34. A computerimplemented simulation and evaluation method according to claim 2 ,further comprising the step of repeating said evolving step (g) to theat least one subsequent health state is responsive to: (1) parallelhealth states of the patient; and (2) a target health state and healthstate combinations that lead to different parallel health states.
 35. Acomputer implemented simulation and evaluation method according to claim30 , wherein the parallel networks of health states comprise: (1) aprimary network including primary health conditions defining a healthdomain; (2) a risk factor network including risk factors for progressionthrough the primary network; and (3) complications attributed totreating the primary health conditions in the primary network.
 36. Acomputer implemented simulation and evaluation method according to claim35 , wherein the parallel networks of health states are generated usingthe following information: (1) how long at least one of the risk factorsexists before influencing a transition between primary health conditionsin the primary network; (2) time required for transitions in the primarynetwork, considering different combinations of the risk factors; and (3)number of transitions the patient is allowed to make between a specifiedhealth state and a normal health state.
 37. A computer simulation andevaluation system for simulating interventions including active andpassive intervention to a patient having a health state by a user, andfor evaluating the interventions responsive to predetermined criteriaand the interventions, comprising: a knowledge database storing patienthealth characteristics including at least one of population, record,agents of change, health states, findings and courses of action; apresentation system providing access to the computer simulation andevaluation system by the user; and a patient simulation system adaptedto be connectable to said presentation system and said knowledgedatabase, said patient simulation system performing the functions: (a)defining a test area and selecting genetic information of the patientresponsive to the test area and the knowledge database; (b) generating apatient history responsive to the test area and the genetic information;(c) receiving at least one intervention input by the user; and (d)evaluating the user responsive to the at least one intervention input bythe user and the predetermined criteria.
 38. A computer readabletangible medium storing instructions for implementing a process drivenby a computer, the process simulating interventions initiated by a user,the interventions including active and passive interventions to apatient having a health state, and the process evaluating theinterventions responsive to predetermined criteria and theinterventions, the instructions comprising the steps of: (a) accessingthe computer implemented simulation and evaluation method by the user;(b) defining a test area to evaluate the user by the computerimplemented simulation and evaluation method responsive to at least oneof predetermined criteria and a user profile; (c) selecting geneticinformation of the patient responsive to the test area; (d) generating apatient history responsive to the test area and the genetic information;(e) receiving at least one intervention input by the user; and (f)evaluating the user responsive to the at least one intervention input bythe user and the predetermined criteria.
 39. A computer implementedsimulation and evaluation method simulates interventions to a patient bya user, and evaluates the interventions responsive to predeterminedcriteria and the interventions, said method comprising the steps ofdefining a test area to evaluate the user responsive to at least one ofpredetermined criteria and a user profile, selecting genetic informationof the patient responsive to the test area, generating a patient historyresponsive to the test area and the genetic information, receiving atleast one intervention input by the user, and evaluating the userresponsive to the at least one intervention and the predeterminedcriteria.